2024-03-18T15:18:44.896 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-18T15:18:44.896 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_pdonnell@teuthology --description fs/shell/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/ubuntu_latest mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/cephfs-shell} --archive /home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608878 --verbose --name pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi -- /home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608878/orig.config.yaml 2024-03-18T15:18:44.937 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-18T15:18:45.016 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608878 branch: wip-batrick-testing-20240314.183243 description: fs/shell/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/ubuntu_latest mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/cephfs-shell} email: pdonnell@redhat.com first_in_suite: false job_id: '7608878' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-batrick-testing-20240314.183243 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - mons down, quorum - out of quorum - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-batrick-testing-20240314.183243 sha1: 4e19f704afd9957ea657ab858f9ecf76401fef6f owner: scheduled_pdonnell@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - osd.4 - osd.5 - osd.6 - osd.7 seed: 4770 sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af sleep_before_teardown: 0 subset: 498/512 suite: fs suite_branch: wip-batrick-testing-20240314.183243 suite_path: /home/teuthworker/src/github.com_batrick_ceph_4e19f704afd9957ea657ab858f9ecf76401fef6f/qa suite_relpath: qa suite_repo: https://github.com/batrick/ceph suite_sha1: 4e19f704afd9957ea657ab858f9ecf76401fef6f syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi005.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBeylqcswqUr05yIIhATKJxEzhYbd6C3wDv2PdDWT4RmCi7iiC/LBqC9AtG2GTM3aparYmIWuzcG5frRFfBO4DI= smithi080.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHNmff7IMarXwxiL+qz9KB3L6ZwbgMuqx413Lxw7UxAhVWj+9yFYOxhyKc9j8ckv7x6x9TGTIXs7Jwujqc59FpQ= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_cephfs_shell teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-18_14:19:46 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-18T15:18:45.017 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_batrick_ceph_4e19f704afd9957ea657ab858f9ecf76401fef6f/qa; will attempt to use it 2024-03-18T15:18:45.017 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_batrick_ceph_4e19f704afd9957ea657ab858f9ecf76401fef6f/qa/tasks 2024-03-18T15:18:45.017 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-18T15:18:45.019 INFO:teuthology.task.internal:Checking packages... 2024-03-18T15:18:45.041 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af' 2024-03-18T15:18:45.041 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-18T15:18:45.041 INFO:teuthology.packaging:ref: None 2024-03-18T15:18:45.041 INFO:teuthology.packaging:tag: None 2024-03-18T15:18:45.041 INFO:teuthology.packaging:branch: wip-batrick-testing-20240314.183243 2024-03-18T15:18:45.041 INFO:teuthology.packaging:sha1: 3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-18T15:18:45.041 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-batrick-testing-20240314.183243 2024-03-18T15:18:45.265 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2162-g3d78f5ba-1jammy 2024-03-18T15:18:45.267 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-18T15:18:45.273 INFO:teuthology.task.internal:no buildpackages task found 2024-03-18T15:18:45.273 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-18T15:18:45.278 INFO:teuthology.task.internal:Saving configuration 2024-03-18T15:18:45.292 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-18T15:18:45.298 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-18T15:18:45.320 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi005.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608878', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-18 15:12:57.349464', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFXvPUhO8ttskDsa1RrRlWE0Cs7g6y3uC9Jzwl6Hslzy'} 2024-03-18T15:18:45.340 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi080.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608878', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-18 15:12:57.347407', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICIp9wU5/ewYuVB3AjQOYSahJzZLpfNBL59Wz/4L3mTz'} 2024-03-18T15:18:45.340 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-18T15:18:45.346 INFO:teuthology.task.internal:roles: ubuntu@smithi005.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-03-18T15:18:45.346 INFO:teuthology.task.internal:roles: ubuntu@smithi080.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2024-03-18T15:18:45.346 INFO:teuthology.run_tasks:Running task console_log... 2024-03-18T15:18:45.415 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4c188313a0>, signals=[15]) 2024-03-18T15:18:45.415 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-18T15:18:45.421 INFO:teuthology.task.internal:Opening connections... 2024-03-18T15:18:45.421 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi005.front.sepia.ceph.com 2024-03-18T15:18:45.423 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T15:18:45.502 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi080.front.sepia.ceph.com 2024-03-18T15:18:45.503 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T15:18:45.582 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-18T15:18:45.589 DEBUG:teuthology.orchestra.run.smithi005:> uname -m 2024-03-18T15:18:45.595 INFO:teuthology.orchestra.run.smithi005.stdout:x86_64 2024-03-18T15:18:45.595 DEBUG:teuthology.orchestra.run.smithi005:> cat /etc/os-release 2024-03-18T15:18:45.643 INFO:teuthology.orchestra.run.smithi005.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-03-18T15:18:45.643 INFO:teuthology.orchestra.run.smithi005.stdout:NAME="Ubuntu" 2024-03-18T15:18:45.643 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION_ID="22.04" 2024-03-18T15:18:45.643 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-03-18T15:18:45.643 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION_CODENAME=jammy 2024-03-18T15:18:45.643 INFO:teuthology.orchestra.run.smithi005.stdout:ID=ubuntu 2024-03-18T15:18:45.643 INFO:teuthology.orchestra.run.smithi005.stdout:ID_LIKE=debian 2024-03-18T15:18:45.643 INFO:teuthology.orchestra.run.smithi005.stdout:HOME_URL="https://www.ubuntu.com/" 2024-03-18T15:18:45.643 INFO:teuthology.orchestra.run.smithi005.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-03-18T15:18:45.643 INFO:teuthology.orchestra.run.smithi005.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-03-18T15:18:45.643 INFO:teuthology.orchestra.run.smithi005.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-03-18T15:18:45.643 INFO:teuthology.orchestra.run.smithi005.stdout:UBUNTU_CODENAME=jammy 2024-03-18T15:18:45.644 INFO:teuthology.lock.ops:Updating smithi005.front.sepia.ceph.com on lock server 2024-03-18T15:18:45.666 DEBUG:teuthology.orchestra.run.smithi080:> uname -m 2024-03-18T15:18:45.671 INFO:teuthology.orchestra.run.smithi080.stdout:x86_64 2024-03-18T15:18:45.671 DEBUG:teuthology.orchestra.run.smithi080:> cat /etc/os-release 2024-03-18T15:18:45.717 INFO:teuthology.orchestra.run.smithi080.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-03-18T15:18:45.718 INFO:teuthology.orchestra.run.smithi080.stdout:NAME="Ubuntu" 2024-03-18T15:18:45.718 INFO:teuthology.orchestra.run.smithi080.stdout:VERSION_ID="22.04" 2024-03-18T15:18:45.718 INFO:teuthology.orchestra.run.smithi080.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-03-18T15:18:45.718 INFO:teuthology.orchestra.run.smithi080.stdout:VERSION_CODENAME=jammy 2024-03-18T15:18:45.718 INFO:teuthology.orchestra.run.smithi080.stdout:ID=ubuntu 2024-03-18T15:18:45.718 INFO:teuthology.orchestra.run.smithi080.stdout:ID_LIKE=debian 2024-03-18T15:18:45.718 INFO:teuthology.orchestra.run.smithi080.stdout:HOME_URL="https://www.ubuntu.com/" 2024-03-18T15:18:45.718 INFO:teuthology.orchestra.run.smithi080.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-03-18T15:18:45.718 INFO:teuthology.orchestra.run.smithi080.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-03-18T15:18:45.718 INFO:teuthology.orchestra.run.smithi080.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-03-18T15:18:45.718 INFO:teuthology.orchestra.run.smithi080.stdout:UBUNTU_CODENAME=jammy 2024-03-18T15:18:45.719 INFO:teuthology.lock.ops:Updating smithi080.front.sepia.ceph.com on lock server 2024-03-18T15:18:45.741 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-18T15:18:45.750 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-18T15:18:45.755 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-18T15:18:45.755 DEBUG:teuthology.orchestra.run.smithi005:> test '!' -e /home/ubuntu/cephtest 2024-03-18T15:18:45.757 DEBUG:teuthology.orchestra.run.smithi080:> test '!' -e /home/ubuntu/cephtest 2024-03-18T15:18:45.769 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-18T15:18:45.775 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-18T15:18:45.775 DEBUG:teuthology.orchestra.run.smithi005:> test -z $(ls -A /var/lib/ceph) 2024-03-18T15:18:45.804 DEBUG:teuthology.orchestra.run.smithi080:> test -z $(ls -A /var/lib/ceph) 2024-03-18T15:18:45.824 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-18T15:18:45.875 INFO:teuthology.run_tasks:Running task kernel... 2024-03-18T15:18:45.889 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-03-18T15:18:45.889 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-03-18T15:18:45.889 DEBUG:teuthology.orchestra.run.smithi005:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-18T15:18:45.890 DEBUG:teuthology.orchestra.run.smithi080:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-18T15:18:45.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T15:18:45.894 DEBUG:teuthology.orchestra.run.smithi005:> uname -r 2024-03-18T15:18:45.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T15:18:45.895 DEBUG:teuthology.orchestra.run.smithi080:> uname -r 2024-03-18T15:18:45.942 INFO:teuthology.orchestra.run.smithi080.stdout:5.15.0-56-generic 2024-03-18T15:18:45.942 INFO:teuthology.task.kernel:Running kernel on smithi080: 5.15.0-56-generic 2024-03-18T15:18:45.942 DEBUG:teuthology.orchestra.run.smithi080:> sudo apt-get clean 2024-03-18T15:18:45.943 INFO:teuthology.orchestra.run.smithi005.stdout:5.15.0-56-generic 2024-03-18T15:18:45.943 INFO:teuthology.task.kernel:Running kernel on smithi005: 5.15.0-56-generic 2024-03-18T15:18:45.943 DEBUG:teuthology.orchestra.run.smithi005:> sudo apt-get clean 2024-03-18T15:18:46.077 DEBUG:teuthology.orchestra.run.smithi005:> sudo apt-get update 2024-03-18T15:18:46.101 DEBUG:teuthology.orchestra.run.smithi080:> sudo apt-get update 2024-03-18T15:18:46.322 INFO:teuthology.orchestra.run.smithi005.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-18T15:18:46.335 INFO:teuthology.orchestra.run.smithi080.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-18T15:18:46.376 INFO:teuthology.orchestra.run.smithi005.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-03-18T15:18:46.380 INFO:teuthology.orchestra.run.smithi005.stdout:Get:3 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-03-18T15:18:46.381 INFO:teuthology.orchestra.run.smithi080.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-03-18T15:18:46.381 INFO:teuthology.orchestra.run.smithi080.stdout:Get:3 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-03-18T15:18:46.560 INFO:teuthology.orchestra.run.smithi005.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-03-18T15:18:46.572 INFO:teuthology.orchestra.run.smithi080.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-03-18T15:18:46.649 INFO:teuthology.orchestra.run.smithi005.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,490 kB] 2024-03-18T15:18:46.665 INFO:teuthology.orchestra.run.smithi080.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [592 kB] 2024-03-18T15:18:46.799 INFO:teuthology.orchestra.run.smithi080.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,490 kB] 2024-03-18T15:18:46.842 INFO:teuthology.orchestra.run.smithi080.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [421 kB] 2024-03-18T15:18:46.848 INFO:teuthology.orchestra.run.smithi005.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [592 kB] 2024-03-18T15:18:46.848 INFO:teuthology.orchestra.run.smithi005.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,246 kB] 2024-03-18T15:18:46.854 INFO:teuthology.orchestra.run.smithi005.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [288 kB] 2024-03-18T15:18:46.866 INFO:teuthology.orchestra.run.smithi005.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-03-18T15:18:46.867 INFO:teuthology.orchestra.run.smithi005.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,605 kB] 2024-03-18T15:18:46.892 INFO:teuthology.orchestra.run.smithi080.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [288 kB] 2024-03-18T15:18:46.907 INFO:teuthology.orchestra.run.smithi080.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-03-18T15:18:46.972 INFO:teuthology.orchestra.run.smithi005.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [35.3 kB] 2024-03-18T15:18:46.974 INFO:teuthology.orchestra.run.smithi005.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [268 kB] 2024-03-18T15:18:46.978 INFO:teuthology.orchestra.run.smithi080.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,605 kB] 2024-03-18T15:18:46.980 INFO:teuthology.orchestra.run.smithi005.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-03-18T15:18:46.980 INFO:teuthology.orchestra.run.smithi005.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,057 kB] 2024-03-18T15:18:47.002 INFO:teuthology.orchestra.run.smithi005.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [695 kB] 2024-03-18T15:18:47.006 INFO:teuthology.orchestra.run.smithi080.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [35.3 kB] 2024-03-18T15:18:47.007 INFO:teuthology.orchestra.run.smithi080.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [268 kB] 2024-03-18T15:18:47.012 INFO:teuthology.orchestra.run.smithi080.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-03-18T15:18:47.012 INFO:teuthology.orchestra.run.smithi080.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,057 kB] 2024-03-18T15:18:47.014 INFO:teuthology.orchestra.run.smithi005.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [239 kB] 2024-03-18T15:18:47.017 INFO:teuthology.orchestra.run.smithi005.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-03-18T15:18:47.018 INFO:teuthology.orchestra.run.smithi005.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,160 B] 2024-03-18T15:18:47.018 INFO:teuthology.orchestra.run.smithi005.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.1 kB] 2024-03-18T15:18:47.019 INFO:teuthology.orchestra.run.smithi005.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.1 kB] 2024-03-18T15:18:47.030 INFO:teuthology.orchestra.run.smithi080.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [695 kB] 2024-03-18T15:18:47.040 INFO:teuthology.orchestra.run.smithi080.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [239 kB] 2024-03-18T15:18:47.049 INFO:teuthology.orchestra.run.smithi080.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-03-18T15:18:47.050 INFO:teuthology.orchestra.run.smithi005.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-03-18T15:18:47.051 INFO:teuthology.orchestra.run.smithi005.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.2 kB] 2024-03-18T15:18:47.052 INFO:teuthology.orchestra.run.smithi005.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-03-18T15:18:47.053 INFO:teuthology.orchestra.run.smithi005.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-03-18T15:18:47.071 INFO:teuthology.orchestra.run.smithi080.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.1 kB] 2024-03-18T15:18:47.072 INFO:teuthology.orchestra.run.smithi080.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,160 B] 2024-03-18T15:18:47.074 INFO:teuthology.orchestra.run.smithi080.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.1 kB] 2024-03-18T15:18:47.074 INFO:teuthology.orchestra.run.smithi080.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-03-18T15:18:47.075 INFO:teuthology.orchestra.run.smithi080.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.2 kB] 2024-03-18T15:18:47.076 INFO:teuthology.orchestra.run.smithi080.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-03-18T15:18:47.077 INFO:teuthology.orchestra.run.smithi080.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-03-18T15:18:47.077 INFO:teuthology.orchestra.run.smithi080.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-03-18T15:18:47.084 INFO:teuthology.orchestra.run.smithi005.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-03-18T15:18:47.084 INFO:teuthology.orchestra.run.smithi005.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.4 kB] 2024-03-18T15:18:47.084 INFO:teuthology.orchestra.run.smithi005.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.2 kB] 2024-03-18T15:18:47.085 INFO:teuthology.orchestra.run.smithi005.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-03-18T15:18:47.085 INFO:teuthology.orchestra.run.smithi005.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-03-18T15:18:47.108 INFO:teuthology.orchestra.run.smithi080.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.4 kB] 2024-03-18T15:18:47.109 INFO:teuthology.orchestra.run.smithi080.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.2 kB] 2024-03-18T15:18:47.110 INFO:teuthology.orchestra.run.smithi080.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-03-18T15:18:47.110 INFO:teuthology.orchestra.run.smithi080.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-03-18T15:18:47.118 INFO:teuthology.orchestra.run.smithi080.stdout:Get:30 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,246 kB] 2024-03-18T15:18:47.227 INFO:teuthology.orchestra.run.smithi005.stdout:Get:30 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [421 kB] 2024-03-18T15:18:47.245 INFO:teuthology.orchestra.run.smithi005.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [225 kB] 2024-03-18T15:18:47.255 INFO:teuthology.orchestra.run.smithi005.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-03-18T15:18:47.256 INFO:teuthology.orchestra.run.smithi005.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [33.7 kB] 2024-03-18T15:18:47.257 INFO:teuthology.orchestra.run.smithi005.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,531 kB] 2024-03-18T15:18:47.283 INFO:teuthology.orchestra.run.smithi080.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [225 kB] 2024-03-18T15:18:47.296 INFO:teuthology.orchestra.run.smithi080.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-03-18T15:18:47.296 INFO:teuthology.orchestra.run.smithi080.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [33.7 kB] 2024-03-18T15:18:47.298 INFO:teuthology.orchestra.run.smithi080.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,531 kB] 2024-03-18T15:18:47.348 INFO:teuthology.orchestra.run.smithi005.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [255 kB] 2024-03-18T15:18:47.358 INFO:teuthology.orchestra.run.smithi005.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-03-18T15:18:47.359 INFO:teuthology.orchestra.run.smithi005.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [599 kB] 2024-03-18T15:18:47.377 INFO:teuthology.orchestra.run.smithi080.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [255 kB] 2024-03-18T15:18:47.381 INFO:teuthology.orchestra.run.smithi005.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [851 kB] 2024-03-18T15:18:47.385 INFO:teuthology.orchestra.run.smithi080.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-03-18T15:18:47.385 INFO:teuthology.orchestra.run.smithi080.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [599 kB] 2024-03-18T15:18:47.402 INFO:teuthology.orchestra.run.smithi080.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [851 kB] 2024-03-18T15:18:47.567 INFO:teuthology.orchestra.run.smithi005.stdout:Get:39 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [162 kB] 2024-03-18T15:18:47.587 INFO:teuthology.orchestra.run.smithi080.stdout:Get:39 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [162 kB] 2024-03-18T15:18:47.964 INFO:teuthology.orchestra.run.smithi005.stdout:Get:40 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-03-18T15:18:47.970 INFO:teuthology.orchestra.run.smithi005.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-03-18T15:18:47.971 INFO:teuthology.orchestra.run.smithi005.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.1 kB] 2024-03-18T15:18:47.982 INFO:teuthology.orchestra.run.smithi080.stdout:Get:40 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-03-18T15:18:47.985 INFO:teuthology.orchestra.run.smithi005.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,476 B] 2024-03-18T15:18:47.988 INFO:teuthology.orchestra.run.smithi005.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-03-18T15:18:47.989 INFO:teuthology.orchestra.run.smithi080.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.1 kB] 2024-03-18T15:18:48.002 INFO:teuthology.orchestra.run.smithi080.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-03-18T15:18:48.004 INFO:teuthology.orchestra.run.smithi080.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,476 B] 2024-03-18T15:18:48.006 INFO:teuthology.orchestra.run.smithi080.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-03-18T15:18:52.698 INFO:teuthology.orchestra.run.smithi080.stdout:Fetched 12.3 MB in 2s (6,133 kB/s) 2024-03-18T15:18:52.927 INFO:teuthology.orchestra.run.smithi005.stdout:Fetched 12.3 MB in 2s (5,770 kB/s) 2024-03-18T15:18:54.065 INFO:teuthology.orchestra.run.smithi080.stdout:Reading package lists... 2024-03-18T15:18:54.083 DEBUG:teuthology.orchestra.run.smithi080:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-03-18T15:18:54.142 INFO:teuthology.orchestra.run.smithi080.stdout:Reading package lists... 2024-03-18T15:18:54.343 INFO:teuthology.orchestra.run.smithi080.stdout:Building dependency tree... 2024-03-18T15:18:54.344 INFO:teuthology.orchestra.run.smithi080.stdout:Reading state information... 2024-03-18T15:18:54.348 INFO:teuthology.orchestra.run.smithi005.stdout:Reading package lists... 2024-03-18T15:18:54.366 DEBUG:teuthology.orchestra.run.smithi005:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-03-18T15:18:54.417 INFO:teuthology.orchestra.run.smithi005.stdout:Reading package lists... 2024-03-18T15:18:54.505 INFO:teuthology.orchestra.run.smithi080.stdout:The following packages were automatically installed and are no longer required: 2024-03-18T15:18:54.505 INFO:teuthology.orchestra.run.smithi080.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-18T15:18:54.505 INFO:teuthology.orchestra.run.smithi080.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-18T15:18:54.506 INFO:teuthology.orchestra.run.smithi080.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-18T15:18:54.506 INFO:teuthology.orchestra.run.smithi080.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-03-18T15:18:54.506 INFO:teuthology.orchestra.run.smithi080.stdout: python2.7-minimal 2024-03-18T15:18:54.506 INFO:teuthology.orchestra.run.smithi080.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-18T15:18:54.507 INFO:teuthology.orchestra.run.smithi080.stdout:The following additional packages will be installed: 2024-03-18T15:18:54.507 INFO:teuthology.orchestra.run.smithi080.stdout: linux-generic linux-headers-5.15.0-101 linux-headers-5.15.0-101-generic 2024-03-18T15:18:54.507 INFO:teuthology.orchestra.run.smithi080.stdout: linux-headers-generic linux-image-5.15.0-101-generic 2024-03-18T15:18:54.508 INFO:teuthology.orchestra.run.smithi080.stdout: linux-modules-5.15.0-101-generic linux-modules-extra-5.15.0-101-generic 2024-03-18T15:18:54.509 INFO:teuthology.orchestra.run.smithi080.stdout:Suggested packages: 2024-03-18T15:18:54.509 INFO:teuthology.orchestra.run.smithi080.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-03-18T15:18:54.509 INFO:teuthology.orchestra.run.smithi080.stdout:Recommended packages: 2024-03-18T15:18:54.509 INFO:teuthology.orchestra.run.smithi080.stdout: thermald 2024-03-18T15:18:54.551 INFO:teuthology.orchestra.run.smithi080.stdout:The following NEW packages will be installed: 2024-03-18T15:18:54.551 INFO:teuthology.orchestra.run.smithi080.stdout: linux-headers-5.15.0-101 linux-headers-5.15.0-101-generic 2024-03-18T15:18:54.551 INFO:teuthology.orchestra.run.smithi080.stdout: linux-image-5.15.0-101-generic linux-modules-5.15.0-101-generic 2024-03-18T15:18:54.552 INFO:teuthology.orchestra.run.smithi080.stdout: linux-modules-extra-5.15.0-101-generic 2024-03-18T15:18:54.553 INFO:teuthology.orchestra.run.smithi080.stdout:The following packages will be upgraded: 2024-03-18T15:18:54.553 INFO:teuthology.orchestra.run.smithi080.stdout: linux-generic linux-headers-generic linux-image-generic 2024-03-18T15:18:54.596 INFO:teuthology.orchestra.run.smithi005.stdout:Building dependency tree... 2024-03-18T15:18:54.597 INFO:teuthology.orchestra.run.smithi005.stdout:Reading state information... 2024-03-18T15:18:54.732 INFO:teuthology.orchestra.run.smithi080.stdout:3 upgraded, 5 newly installed, 0 to remove and 298 not upgraded. 2024-03-18T15:18:54.732 INFO:teuthology.orchestra.run.smithi080.stdout:Need to get 113 MB of archives. 2024-03-18T15:18:54.732 INFO:teuthology.orchestra.run.smithi080.stdout:After this operation, 582 MB of additional disk space will be used. 2024-03-18T15:18:54.732 INFO:teuthology.orchestra.run.smithi080.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-101-generic amd64 5.15.0-101.111 [22.6 MB] 2024-03-18T15:18:54.754 INFO:teuthology.orchestra.run.smithi005.stdout:The following packages were automatically installed and are no longer required: 2024-03-18T15:18:54.754 INFO:teuthology.orchestra.run.smithi005.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-18T15:18:54.754 INFO:teuthology.orchestra.run.smithi005.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-18T15:18:54.755 INFO:teuthology.orchestra.run.smithi005.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-18T15:18:54.755 INFO:teuthology.orchestra.run.smithi005.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-03-18T15:18:54.755 INFO:teuthology.orchestra.run.smithi005.stdout: python2.7-minimal 2024-03-18T15:18:54.755 INFO:teuthology.orchestra.run.smithi005.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-18T15:18:54.756 INFO:teuthology.orchestra.run.smithi005.stdout:The following additional packages will be installed: 2024-03-18T15:18:54.756 INFO:teuthology.orchestra.run.smithi005.stdout: linux-generic linux-headers-5.15.0-101 linux-headers-5.15.0-101-generic 2024-03-18T15:18:54.756 INFO:teuthology.orchestra.run.smithi005.stdout: linux-headers-generic linux-image-5.15.0-101-generic 2024-03-18T15:18:54.756 INFO:teuthology.orchestra.run.smithi005.stdout: linux-modules-5.15.0-101-generic linux-modules-extra-5.15.0-101-generic 2024-03-18T15:18:54.757 INFO:teuthology.orchestra.run.smithi005.stdout:Suggested packages: 2024-03-18T15:18:54.757 INFO:teuthology.orchestra.run.smithi005.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-03-18T15:18:54.757 INFO:teuthology.orchestra.run.smithi005.stdout:Recommended packages: 2024-03-18T15:18:54.758 INFO:teuthology.orchestra.run.smithi005.stdout: thermald 2024-03-18T15:18:54.803 INFO:teuthology.orchestra.run.smithi005.stdout:The following NEW packages will be installed: 2024-03-18T15:18:54.803 INFO:teuthology.orchestra.run.smithi005.stdout: linux-headers-5.15.0-101 linux-headers-5.15.0-101-generic 2024-03-18T15:18:54.803 INFO:teuthology.orchestra.run.smithi005.stdout: linux-image-5.15.0-101-generic linux-modules-5.15.0-101-generic 2024-03-18T15:18:54.804 INFO:teuthology.orchestra.run.smithi005.stdout: linux-modules-extra-5.15.0-101-generic 2024-03-18T15:18:54.805 INFO:teuthology.orchestra.run.smithi005.stdout:The following packages will be upgraded: 2024-03-18T15:18:54.805 INFO:teuthology.orchestra.run.smithi005.stdout: linux-generic linux-headers-generic linux-image-generic 2024-03-18T15:18:54.985 INFO:teuthology.orchestra.run.smithi005.stdout:3 upgraded, 5 newly installed, 0 to remove and 298 not upgraded. 2024-03-18T15:18:54.985 INFO:teuthology.orchestra.run.smithi005.stdout:Need to get 113 MB of archives. 2024-03-18T15:18:54.985 INFO:teuthology.orchestra.run.smithi005.stdout:After this operation, 582 MB of additional disk space will be used. 2024-03-18T15:18:54.985 INFO:teuthology.orchestra.run.smithi005.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-101-generic amd64 5.15.0-101.111 [22.6 MB] 2024-03-18T15:18:56.250 INFO:teuthology.orchestra.run.smithi080.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-101-generic amd64 5.15.0-101.111 [11.5 MB] 2024-03-18T15:18:56.378 INFO:teuthology.orchestra.run.smithi005.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-101-generic amd64 5.15.0-101.111 [11.5 MB] 2024-03-18T15:18:56.602 INFO:teuthology.orchestra.run.smithi080.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-101-generic amd64 5.15.0-101.111 [63.9 MB] 2024-03-18T15:18:56.793 INFO:teuthology.orchestra.run.smithi005.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-101-generic amd64 5.15.0-101.111 [63.9 MB] 2024-03-18T15:18:58.897 INFO:teuthology.orchestra.run.smithi080.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.101.98 [1,698 B] 2024-03-18T15:18:58.897 INFO:teuthology.orchestra.run.smithi080.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.101.98 [2,486 B] 2024-03-18T15:18:58.897 INFO:teuthology.orchestra.run.smithi080.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-101 all 5.15.0-101.111 [12.3 MB] 2024-03-18T15:18:59.004 INFO:teuthology.orchestra.run.smithi005.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.101.98 [1,698 B] 2024-03-18T15:18:59.004 INFO:teuthology.orchestra.run.smithi005.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.101.98 [2,486 B] 2024-03-18T15:18:59.004 INFO:teuthology.orchestra.run.smithi005.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-101 all 5.15.0-101.111 [12.3 MB] 2024-03-18T15:18:59.411 INFO:teuthology.orchestra.run.smithi080.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-101-generic amd64 5.15.0-101.111 [2,876 kB] 2024-03-18T15:18:59.551 INFO:teuthology.orchestra.run.smithi080.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.101.98 [2,332 B] 2024-03-18T15:18:59.939 INFO:teuthology.orchestra.run.smithi080.stdout:Fetched 113 MB in 5s (22.7 MB/s) 2024-03-18T15:19:00.106 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package linux-modules-5.15.0-101-generic. 2024-03-18T15:19:00.289 INFO:teuthology.orchestra.run.smithi005.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-101-generic amd64 5.15.0-101.111 [2,876 kB] 2024-03-18T15:19:00.438 INFO:teuthology.orchestra.run.smithi005.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.101.98 [2,332 B] 2024-03-18T15:19:00.851 INFO:teuthology.orchestra.run.smithi005.stdout:Fetched 113 MB in 6s (20.1 MB/s) 2024-03-18T15:19:01.031 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package linux-modules-5.15.0-101-generic. 2024-03-18T15:19:02.187 INFO:teuthology.orchestra.run.smithi080.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-03-18T15:19:02.191 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../0-linux-modules-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-03-18T15:19:02.328 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking linux-modules-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-18T15:19:03.011 INFO:teuthology.orchestra.run.smithi005.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-03-18T15:19:03.015 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../0-linux-modules-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-03-18T15:19:03.162 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking linux-modules-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-18T15:19:04.813 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package linux-image-5.15.0-101-generic. 2024-03-18T15:19:04.833 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../1-linux-image-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-03-18T15:19:04.931 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking linux-image-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-18T15:19:05.399 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-101-generic. 2024-03-18T15:19:05.420 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-03-18T15:19:05.457 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking linux-modules-extra-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-18T15:19:05.814 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package linux-image-5.15.0-101-generic. 2024-03-18T15:19:05.835 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../1-linux-image-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-03-18T15:19:05.932 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking linux-image-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-18T15:19:06.409 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-101-generic. 2024-03-18T15:19:06.430 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-03-18T15:19:06.467 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking linux-modules-extra-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-18T15:19:11.917 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../3-linux-generic_5.15.0.101.98_amd64.deb ... 2024-03-18T15:19:12.041 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking linux-generic (5.15.0.101.98) over (5.15.0.56.54) ... 2024-03-18T15:19:12.340 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.101.98_amd64.deb ... 2024-03-18T15:19:12.470 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking linux-image-generic (5.15.0.101.98) over (5.15.0.56.54) ... 2024-03-18T15:19:12.756 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package linux-headers-5.15.0-101. 2024-03-18T15:19:12.778 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../5-linux-headers-5.15.0-101_5.15.0-101.111_all.deb ... 2024-03-18T15:19:12.822 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking linux-headers-5.15.0-101 (5.15.0-101.111) ... 2024-03-18T15:19:13.020 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../3-linux-generic_5.15.0.101.98_amd64.deb ... 2024-03-18T15:19:13.153 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking linux-generic (5.15.0.101.98) over (5.15.0.56.54) ... 2024-03-18T15:19:13.494 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.101.98_amd64.deb ... 2024-03-18T15:19:13.615 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking linux-image-generic (5.15.0.101.98) over (5.15.0.56.54) ... 2024-03-18T15:19:13.910 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package linux-headers-5.15.0-101. 2024-03-18T15:19:13.932 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../5-linux-headers-5.15.0-101_5.15.0-101.111_all.deb ... 2024-03-18T15:19:13.976 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking linux-headers-5.15.0-101 (5.15.0-101.111) ... 2024-03-18T15:19:18.382 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package linux-headers-5.15.0-101-generic. 2024-03-18T15:19:18.418 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../6-linux-headers-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-03-18T15:19:18.456 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking linux-headers-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-18T15:19:19.690 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package linux-headers-5.15.0-101-generic. 2024-03-18T15:19:19.727 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../6-linux-headers-5.15.0-101-generic_5.15.0-101.111_amd64.deb ... 2024-03-18T15:19:19.756 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking linux-headers-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-18T15:19:21.154 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.101.98_amd64.deb ... 2024-03-18T15:19:21.276 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking linux-headers-generic (5.15.0.101.98) over (5.15.0.56.54) ... 2024-03-18T15:19:21.668 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up linux-headers-5.15.0-101 (5.15.0-101.111) ... 2024-03-18T15:19:22.473 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up linux-headers-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-18T15:19:22.599 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up linux-headers-generic (5.15.0.101.98) ... 2024-03-18T15:19:22.728 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up linux-image-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-18T15:19:23.952 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.101.98_amd64.deb ... 2024-03-18T15:19:24.017 INFO:teuthology.orchestra.run.smithi080.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-03-18T15:19:24.017 INFO:teuthology.orchestra.run.smithi080.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-03-18T15:19:24.017 INFO:teuthology.orchestra.run.smithi080.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-101-generic 2024-03-18T15:19:24.017 INFO:teuthology.orchestra.run.smithi080.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-101-generic 2024-03-18T15:19:24.077 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking linux-headers-generic (5.15.0.101.98) over (5.15.0.56.54) ... 2024-03-18T15:19:24.180 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up linux-modules-extra-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-18T15:19:24.444 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up linux-headers-5.15.0-101 (5.15.0-101.111) ... 2024-03-18T15:19:24.557 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up linux-headers-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-18T15:19:24.691 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up linux-headers-generic (5.15.0.101.98) ... 2024-03-18T15:19:24.820 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up linux-image-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-18T15:19:25.589 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up linux-modules-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-18T15:19:26.182 INFO:teuthology.orchestra.run.smithi005.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-03-18T15:19:26.182 INFO:teuthology.orchestra.run.smithi005.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-03-18T15:19:26.182 INFO:teuthology.orchestra.run.smithi005.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-101-generic 2024-03-18T15:19:26.182 INFO:teuthology.orchestra.run.smithi005.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-101-generic 2024-03-18T15:19:26.340 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up linux-modules-extra-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-18T15:19:26.957 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up linux-image-generic (5.15.0.101.98) ... 2024-03-18T15:19:27.083 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up linux-generic (5.15.0.101.98) ... 2024-03-18T15:19:27.209 INFO:teuthology.orchestra.run.smithi080.stdout:Processing triggers for linux-image-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-18T15:19:27.275 INFO:teuthology.orchestra.run.smithi080.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-03-18T15:19:27.275 INFO:teuthology.orchestra.run.smithi080.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-101-generic 2024-03-18T15:19:28.465 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up linux-modules-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-18T15:19:29.892 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up linux-image-generic (5.15.0.101.98) ... 2024-03-18T15:19:30.018 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up linux-generic (5.15.0.101.98) ... 2024-03-18T15:19:30.144 INFO:teuthology.orchestra.run.smithi005.stdout:Processing triggers for linux-image-5.15.0-101-generic (5.15.0-101.111) ... 2024-03-18T15:19:30.233 INFO:teuthology.orchestra.run.smithi005.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-03-18T15:19:30.233 INFO:teuthology.orchestra.run.smithi005.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-101-generic 2024-03-18T15:19:41.575 INFO:teuthology.orchestra.run.smithi080.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-03-18T15:19:41.575 INFO:teuthology.orchestra.run.smithi080.stdout:Sourcing file `/etc/default/grub' 2024-03-18T15:19:41.593 INFO:teuthology.orchestra.run.smithi080.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-18T15:19:41.613 INFO:teuthology.orchestra.run.smithi080.stdout:Generating grub configuration file ... 2024-03-18T15:19:41.895 INFO:teuthology.orchestra.run.smithi080.stdout:Found linux image: /boot/vmlinuz-5.15.0-101-generic 2024-03-18T15:19:41.916 INFO:teuthology.orchestra.run.smithi080.stdout:Found initrd image: /boot/initrd.img-5.15.0-101-generic 2024-03-18T15:19:42.218 INFO:teuthology.orchestra.run.smithi080.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-18T15:19:42.224 INFO:teuthology.orchestra.run.smithi080.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-18T15:19:42.302 INFO:teuthology.orchestra.run.smithi080.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-18T15:19:42.307 INFO:teuthology.orchestra.run.smithi080.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-18T15:19:42.462 INFO:teuthology.orchestra.run.smithi080.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-18T15:19:42.462 INFO:teuthology.orchestra.run.smithi080.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-03-18T15:19:42.462 INFO:teuthology.orchestra.run.smithi080.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-18T15:19:42.503 INFO:teuthology.orchestra.run.smithi080.stdout:done 2024-03-18T15:19:42.966 DEBUG:teuthology.orchestra.run.smithi080:> dpkg -s linux-image-generic 2024-03-18T15:19:42.984 INFO:teuthology.orchestra.run.smithi080.stdout:Package: linux-image-generic 2024-03-18T15:19:42.984 INFO:teuthology.orchestra.run.smithi080.stdout:Status: install ok installed 2024-03-18T15:19:42.984 INFO:teuthology.orchestra.run.smithi080.stdout:Priority: optional 2024-03-18T15:19:42.984 INFO:teuthology.orchestra.run.smithi080.stdout:Section: kernel 2024-03-18T15:19:42.985 INFO:teuthology.orchestra.run.smithi080.stdout:Installed-Size: 21 2024-03-18T15:19:42.985 INFO:teuthology.orchestra.run.smithi080.stdout:Maintainer: Ubuntu Kernel Team 2024-03-18T15:19:42.985 INFO:teuthology.orchestra.run.smithi080.stdout:Architecture: amd64 2024-03-18T15:19:42.985 INFO:teuthology.orchestra.run.smithi080.stdout:Source: linux-meta 2024-03-18T15:19:42.985 INFO:teuthology.orchestra.run.smithi080.stdout:Version: 5.15.0.101.98 2024-03-18T15:19:42.985 INFO:teuthology.orchestra.run.smithi080.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-101), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-03-18T15:19:42.985 INFO:teuthology.orchestra.run.smithi080.stdout:Depends: linux-image-5.15.0-101-generic, linux-modules-extra-5.15.0-101-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-18T15:19:42.985 INFO:teuthology.orchestra.run.smithi080.stdout:Recommends: thermald 2024-03-18T15:19:42.985 INFO:teuthology.orchestra.run.smithi080.stdout:Description: Generic Linux kernel image 2024-03-18T15:19:42.985 INFO:teuthology.orchestra.run.smithi080.stdout: This package will always depend on the latest generic kernel image 2024-03-18T15:19:42.985 INFO:teuthology.orchestra.run.smithi080.stdout: available. 2024-03-18T15:19:42.985 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-101-generic 2024-03-18T15:19:42.985 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-18T15:19:42.986 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-18T15:19:42.986 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-03-18T15:19:42.986 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi080.front.sepia.ceph.com, path=None, version=distro) 2024-03-18T15:19:42.986 DEBUG:teuthology.orchestra.run.smithi080:> sudo apt-get clean 2024-03-18T15:19:43.171 DEBUG:teuthology.orchestra.run.smithi080:> sudo apt-get update 2024-03-18T15:19:43.334 INFO:teuthology.orchestra.run.smithi080.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-18T15:19:43.334 INFO:teuthology.orchestra.run.smithi080.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-18T15:19:43.340 INFO:teuthology.orchestra.run.smithi080.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-18T15:19:43.372 INFO:teuthology.orchestra.run.smithi080.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-18T15:19:44.683 INFO:teuthology.orchestra.run.smithi080.stdout:Reading package lists... 2024-03-18T15:19:44.699 DEBUG:teuthology.orchestra.run.smithi080:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-03-18T15:19:44.751 INFO:teuthology.orchestra.run.smithi080.stdout:Reading package lists... 2024-03-18T15:19:44.807 INFO:teuthology.orchestra.run.smithi005.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-03-18T15:19:44.807 INFO:teuthology.orchestra.run.smithi005.stdout:Sourcing file `/etc/default/grub' 2024-03-18T15:19:44.831 INFO:teuthology.orchestra.run.smithi005.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-18T15:19:44.853 INFO:teuthology.orchestra.run.smithi005.stdout:Generating grub configuration file ... 2024-03-18T15:19:44.925 INFO:teuthology.orchestra.run.smithi080.stdout:Building dependency tree... 2024-03-18T15:19:44.926 INFO:teuthology.orchestra.run.smithi080.stdout:Reading state information... 2024-03-18T15:19:45.076 INFO:teuthology.orchestra.run.smithi080.stdout:linux-image-generic is already the newest version (5.15.0.101.98). 2024-03-18T15:19:45.076 INFO:teuthology.orchestra.run.smithi080.stdout:The following packages were automatically installed and are no longer required: 2024-03-18T15:19:45.076 INFO:teuthology.orchestra.run.smithi080.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-18T15:19:45.076 INFO:teuthology.orchestra.run.smithi080.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-18T15:19:45.076 INFO:teuthology.orchestra.run.smithi080.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-18T15:19:45.076 INFO:teuthology.orchestra.run.smithi080.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-18T15:19:45.076 INFO:teuthology.orchestra.run.smithi080.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-18T15:19:45.076 INFO:teuthology.orchestra.run.smithi080.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-18T15:19:45.118 INFO:teuthology.orchestra.run.smithi080.stdout:0 upgraded, 0 newly installed, 0 to remove and 298 not upgraded. 2024-03-18T15:19:45.119 DEBUG:teuthology.orchestra.run.smithi080:> dpkg -s linux-image-generic 2024-03-18T15:19:45.120 INFO:teuthology.orchestra.run.smithi005.stdout:Found linux image: /boot/vmlinuz-5.15.0-101-generic 2024-03-18T15:19:45.134 INFO:teuthology.orchestra.run.smithi005.stdout:Found initrd image: /boot/initrd.img-5.15.0-101-generic 2024-03-18T15:19:45.175 INFO:teuthology.orchestra.run.smithi080.stdout:Package: linux-image-generic 2024-03-18T15:19:45.175 INFO:teuthology.orchestra.run.smithi080.stdout:Status: install ok installed 2024-03-18T15:19:45.175 INFO:teuthology.orchestra.run.smithi080.stdout:Priority: optional 2024-03-18T15:19:45.176 INFO:teuthology.orchestra.run.smithi080.stdout:Section: kernel 2024-03-18T15:19:45.176 INFO:teuthology.orchestra.run.smithi080.stdout:Installed-Size: 21 2024-03-18T15:19:45.176 INFO:teuthology.orchestra.run.smithi080.stdout:Maintainer: Ubuntu Kernel Team 2024-03-18T15:19:45.176 INFO:teuthology.orchestra.run.smithi080.stdout:Architecture: amd64 2024-03-18T15:19:45.176 INFO:teuthology.orchestra.run.smithi080.stdout:Source: linux-meta 2024-03-18T15:19:45.176 INFO:teuthology.orchestra.run.smithi080.stdout:Version: 5.15.0.101.98 2024-03-18T15:19:45.176 INFO:teuthology.orchestra.run.smithi080.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-101), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-03-18T15:19:45.176 INFO:teuthology.orchestra.run.smithi080.stdout:Depends: linux-image-5.15.0-101-generic, linux-modules-extra-5.15.0-101-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-18T15:19:45.176 INFO:teuthology.orchestra.run.smithi080.stdout:Recommends: thermald 2024-03-18T15:19:45.176 INFO:teuthology.orchestra.run.smithi080.stdout:Description: Generic Linux kernel image 2024-03-18T15:19:45.176 INFO:teuthology.orchestra.run.smithi080.stdout: This package will always depend on the latest generic kernel image 2024-03-18T15:19:45.176 INFO:teuthology.orchestra.run.smithi080.stdout: available. 2024-03-18T15:19:45.176 DEBUG:teuthology.orchestra.run.smithi080:> mktemp 2024-03-18T15:19:45.221 INFO:teuthology.orchestra.run.smithi080.stdout:/tmp/tmp.kC4ZPgpBum 2024-03-18T15:19:45.221 DEBUG:teuthology.orchestra.run.smithi080:> sudo cp /boot/grub/grub.cfg /tmp/tmp.kC4ZPgpBum 2024-03-18T15:19:45.277 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod 0666 /tmp/tmp.kC4ZPgpBum 2024-03-18T15:19:45.395 DEBUG:teuthology.orchestra.remote:smithi080:/tmp/tmp.kC4ZPgpBum is 10KB 2024-03-18T15:19:45.409 DEBUG:teuthology.orchestra.run.smithi080:> rm -fr /tmp/tmp.kC4ZPgpBum 2024-03-18T15:19:45.414 DEBUG:teuthology.orchestra.run.smithi080:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-03-18T15:19:45.469 INFO:teuthology.orchestra.run.smithi005.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-18T15:19:45.469 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-03-18T15:19:45.469 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-03-18T15:19:45.469 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-03-18T15:19:45.474 INFO:teuthology.orchestra.run.smithi005.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-18T15:19:45.537 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-101-generic 2024-03-18T15:19:45.537 DEBUG:teuthology.orchestra.run.smithi080:> sudo update-grub 2024-03-18T15:19:45.541 INFO:teuthology.orchestra.run.smithi005.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-18T15:19:45.547 INFO:teuthology.orchestra.run.smithi005.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-18T15:19:45.736 INFO:teuthology.orchestra.run.smithi005.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-18T15:19:45.736 INFO:teuthology.orchestra.run.smithi005.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-03-18T15:19:45.736 INFO:teuthology.orchestra.run.smithi005.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-18T15:19:45.782 INFO:teuthology.orchestra.run.smithi005.stdout:done 2024-03-18T15:19:46.245 DEBUG:teuthology.orchestra.run.smithi005:> dpkg -s linux-image-generic 2024-03-18T15:19:46.266 INFO:teuthology.orchestra.run.smithi005.stdout:Package: linux-image-generic 2024-03-18T15:19:46.266 INFO:teuthology.orchestra.run.smithi005.stdout:Status: install ok installed 2024-03-18T15:19:46.266 INFO:teuthology.orchestra.run.smithi005.stdout:Priority: optional 2024-03-18T15:19:46.266 INFO:teuthology.orchestra.run.smithi005.stdout:Section: kernel 2024-03-18T15:19:46.266 INFO:teuthology.orchestra.run.smithi005.stdout:Installed-Size: 21 2024-03-18T15:19:46.267 INFO:teuthology.orchestra.run.smithi005.stdout:Maintainer: Ubuntu Kernel Team 2024-03-18T15:19:46.267 INFO:teuthology.orchestra.run.smithi005.stdout:Architecture: amd64 2024-03-18T15:19:46.267 INFO:teuthology.orchestra.run.smithi005.stdout:Source: linux-meta 2024-03-18T15:19:46.267 INFO:teuthology.orchestra.run.smithi005.stdout:Version: 5.15.0.101.98 2024-03-18T15:19:46.267 INFO:teuthology.orchestra.run.smithi005.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-101), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-03-18T15:19:46.267 INFO:teuthology.orchestra.run.smithi005.stdout:Depends: linux-image-5.15.0-101-generic, linux-modules-extra-5.15.0-101-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-18T15:19:46.267 INFO:teuthology.orchestra.run.smithi005.stdout:Recommends: thermald 2024-03-18T15:19:46.267 INFO:teuthology.orchestra.run.smithi005.stdout:Description: Generic Linux kernel image 2024-03-18T15:19:46.267 INFO:teuthology.orchestra.run.smithi005.stdout: This package will always depend on the latest generic kernel image 2024-03-18T15:19:46.267 INFO:teuthology.orchestra.run.smithi005.stdout: available. 2024-03-18T15:19:46.267 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-101-generic 2024-03-18T15:19:46.267 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-18T15:19:46.267 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-18T15:19:46.268 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-03-18T15:19:46.268 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi005.front.sepia.ceph.com, path=None, version=distro) 2024-03-18T15:19:46.268 DEBUG:teuthology.orchestra.run.smithi005:> sudo apt-get clean 2024-03-18T15:19:46.410 DEBUG:teuthology.orchestra.run.smithi005:> sudo apt-get update 2024-03-18T15:19:46.685 INFO:teuthology.orchestra.run.smithi080.stderr:Sourcing file `/etc/default/grub' 2024-03-18T15:19:46.687 INFO:teuthology.orchestra.run.smithi080.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-18T15:19:46.688 INFO:teuthology.orchestra.run.smithi080.stderr:Generating grub configuration file ... 2024-03-18T15:19:46.703 INFO:teuthology.orchestra.run.smithi005.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-18T15:19:46.704 INFO:teuthology.orchestra.run.smithi005.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-18T15:19:46.767 INFO:teuthology.orchestra.run.smithi005.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-18T15:19:46.849 INFO:teuthology.orchestra.run.smithi005.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-18T15:19:46.870 INFO:teuthology.orchestra.run.smithi080.stderr:Found linux image: /boot/vmlinuz-5.15.0-101-generic 2024-03-18T15:19:46.877 INFO:teuthology.orchestra.run.smithi080.stderr:Found initrd image: /boot/initrd.img-5.15.0-101-generic 2024-03-18T15:19:47.112 INFO:teuthology.orchestra.run.smithi080.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-18T15:19:47.117 INFO:teuthology.orchestra.run.smithi080.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-18T15:19:47.156 INFO:teuthology.orchestra.run.smithi080.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-18T15:19:47.160 INFO:teuthology.orchestra.run.smithi080.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-18T15:19:47.297 INFO:teuthology.orchestra.run.smithi080.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-18T15:19:47.297 INFO:teuthology.orchestra.run.smithi080.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-03-18T15:19:47.297 INFO:teuthology.orchestra.run.smithi080.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-18T15:19:47.309 INFO:teuthology.orchestra.run.smithi080.stderr:done 2024-03-18T15:19:47.322 DEBUG:teuthology.orchestra.run.smithi080:> sudo shutdown -r now 2024-03-18T15:19:48.070 INFO:teuthology.orchestra.run.smithi005.stdout:Reading package lists... 2024-03-18T15:19:48.089 DEBUG:teuthology.orchestra.run.smithi005:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-03-18T15:19:48.156 INFO:teuthology.orchestra.run.smithi005.stdout:Reading package lists... 2024-03-18T15:19:48.357 INFO:teuthology.orchestra.run.smithi005.stdout:Building dependency tree... 2024-03-18T15:19:48.359 INFO:teuthology.orchestra.run.smithi005.stdout:Reading state information... 2024-03-18T15:19:48.546 INFO:teuthology.orchestra.run.smithi005.stdout:linux-image-generic is already the newest version (5.15.0.101.98). 2024-03-18T15:19:48.546 INFO:teuthology.orchestra.run.smithi005.stdout:The following packages were automatically installed and are no longer required: 2024-03-18T15:19:48.546 INFO:teuthology.orchestra.run.smithi005.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-18T15:19:48.546 INFO:teuthology.orchestra.run.smithi005.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-18T15:19:48.546 INFO:teuthology.orchestra.run.smithi005.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-18T15:19:48.546 INFO:teuthology.orchestra.run.smithi005.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-18T15:19:48.546 INFO:teuthology.orchestra.run.smithi005.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-18T15:19:48.546 INFO:teuthology.orchestra.run.smithi005.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-18T15:19:48.590 INFO:teuthology.orchestra.run.smithi005.stdout:0 upgraded, 0 newly installed, 0 to remove and 298 not upgraded. 2024-03-18T15:19:48.592 DEBUG:teuthology.orchestra.run.smithi005:> dpkg -s linux-image-generic 2024-03-18T15:19:48.608 INFO:teuthology.orchestra.run.smithi005.stdout:Package: linux-image-generic 2024-03-18T15:19:48.609 INFO:teuthology.orchestra.run.smithi005.stdout:Status: install ok installed 2024-03-18T15:19:48.609 INFO:teuthology.orchestra.run.smithi005.stdout:Priority: optional 2024-03-18T15:19:48.609 INFO:teuthology.orchestra.run.smithi005.stdout:Section: kernel 2024-03-18T15:19:48.609 INFO:teuthology.orchestra.run.smithi005.stdout:Installed-Size: 21 2024-03-18T15:19:48.609 INFO:teuthology.orchestra.run.smithi005.stdout:Maintainer: Ubuntu Kernel Team 2024-03-18T15:19:48.609 INFO:teuthology.orchestra.run.smithi005.stdout:Architecture: amd64 2024-03-18T15:19:48.609 INFO:teuthology.orchestra.run.smithi005.stdout:Source: linux-meta 2024-03-18T15:19:48.609 INFO:teuthology.orchestra.run.smithi005.stdout:Version: 5.15.0.101.98 2024-03-18T15:19:48.609 INFO:teuthology.orchestra.run.smithi005.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-101), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-03-18T15:19:48.609 INFO:teuthology.orchestra.run.smithi005.stdout:Depends: linux-image-5.15.0-101-generic, linux-modules-extra-5.15.0-101-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-18T15:19:48.609 INFO:teuthology.orchestra.run.smithi005.stdout:Recommends: thermald 2024-03-18T15:19:48.609 INFO:teuthology.orchestra.run.smithi005.stdout:Description: Generic Linux kernel image 2024-03-18T15:19:48.610 INFO:teuthology.orchestra.run.smithi005.stdout: This package will always depend on the latest generic kernel image 2024-03-18T15:19:48.610 INFO:teuthology.orchestra.run.smithi005.stdout: available. 2024-03-18T15:19:48.610 DEBUG:teuthology.orchestra.run.smithi005:> mktemp 2024-03-18T15:19:48.658 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/tmp.XFIeCQOrde 2024-03-18T15:19:48.658 DEBUG:teuthology.orchestra.run.smithi005:> sudo cp /boot/grub/grub.cfg /tmp/tmp.XFIeCQOrde 2024-03-18T15:19:48.731 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0666 /tmp/tmp.XFIeCQOrde 2024-03-18T15:19:48.864 DEBUG:teuthology.orchestra.remote:smithi005:/tmp/tmp.XFIeCQOrde is 10KB 2024-03-18T15:19:48.878 DEBUG:teuthology.orchestra.run.smithi005:> rm -fr /tmp/tmp.XFIeCQOrde 2024-03-18T15:19:48.883 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-03-18T15:19:48.942 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-03-18T15:19:48.943 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-03-18T15:19:48.943 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-03-18T15:19:49.023 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-101-generic 2024-03-18T15:19:49.023 DEBUG:teuthology.orchestra.run.smithi005:> sudo update-grub 2024-03-18T15:19:50.351 INFO:teuthology.orchestra.run.smithi005.stderr:Sourcing file `/etc/default/grub' 2024-03-18T15:19:50.352 INFO:teuthology.orchestra.run.smithi005.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-18T15:19:50.355 INFO:teuthology.orchestra.run.smithi005.stderr:Generating grub configuration file ... 2024-03-18T15:19:50.534 INFO:teuthology.orchestra.run.smithi005.stderr:Found linux image: /boot/vmlinuz-5.15.0-101-generic 2024-03-18T15:19:50.538 INFO:teuthology.orchestra.run.smithi005.stderr:Found initrd image: /boot/initrd.img-5.15.0-101-generic 2024-03-18T15:19:50.804 INFO:teuthology.orchestra.run.smithi005.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-18T15:19:50.810 INFO:teuthology.orchestra.run.smithi005.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-18T15:19:50.855 INFO:teuthology.orchestra.run.smithi005.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-18T15:19:50.860 INFO:teuthology.orchestra.run.smithi005.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-18T15:19:50.973 INFO:teuthology.orchestra.run.smithi005.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-18T15:19:50.973 INFO:teuthology.orchestra.run.smithi005.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-03-18T15:19:50.973 INFO:teuthology.orchestra.run.smithi005.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-18T15:19:50.983 INFO:teuthology.orchestra.run.smithi005.stderr:done 2024-03-18T15:19:50.985 DEBUG:teuthology.orchestra.run.smithi005:> sudo shutdown -r now 2024-03-18T15:20:17.352 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-03-18T15:20:17.352 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi080.front.sepia.ceph.com' 2024-03-18T15:20:17.353 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T15:20:20.991 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-03-18T15:20:20.991 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-03-18T15:20:20.992 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T15:20:39.454 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.5 2024-03-18T15:20:48.455 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-03-18T15:20:48.456 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T15:20:51.998 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2024-03-18T15:21:01.006 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi080.front.sepia.ceph.com' 2024-03-18T15:21:01.007 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T15:21:19.390 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2024-03-18T15:21:31.403 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi080.front.sepia.ceph.com' 2024-03-18T15:21:31.404 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T15:21:31.407 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2024-03-18T15:21:46.422 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi080.front.sepia.ceph.com' 2024-03-18T15:21:46.423 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T15:21:46.699 DEBUG:teuthology.orchestra.run.smithi080:> true 2024-03-18T15:21:47.298 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi080.front.sepia.ceph.com' 2024-03-18T15:21:47.298 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-101-generic"... 2024-03-18T15:21:47.298 DEBUG:teuthology.orchestra.run.smithi080:> uname -r 2024-03-18T15:21:47.346 INFO:teuthology.orchestra.run.smithi080.stdout:5.15.0-101-generic 2024-03-18T15:21:47.346 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-101-generic vs 5.15.0-101-generic 2024-03-18T15:21:47.346 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-18T15:21:47.346 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-03-18T15:21:48.347 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-03-18T15:21:48.348 DEBUG:teuthology.orchestra.run.smithi080:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-18T15:21:48.459 DEBUG:teuthology.orchestra.remote:timed out 2024-03-18T15:21:48.693 INFO:teuthology.orchestra.run.smithi080.stdout:ttyS1 2024-03-18T15:21:48.720 DEBUG:teuthology.parallel:result is None 2024-03-18T15:22:00.460 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-03-18T15:22:00.460 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T15:22:00.670 DEBUG:teuthology.orchestra.run.smithi005:> true 2024-03-18T15:22:01.335 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-03-18T15:22:01.335 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-101-generic"... 2024-03-18T15:22:01.335 DEBUG:teuthology.orchestra.run.smithi005:> uname -r 2024-03-18T15:22:01.383 INFO:teuthology.orchestra.run.smithi005.stdout:5.15.0-101-generic 2024-03-18T15:22:01.384 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-101-generic vs 5.15.0-101-generic 2024-03-18T15:22:01.384 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-18T15:22:01.384 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-03-18T15:22:02.385 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-03-18T15:22:02.385 DEBUG:teuthology.orchestra.run.smithi005:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-18T15:22:02.452 INFO:teuthology.orchestra.run.smithi005.stdout:ttyS1 2024-03-18T15:22:02.473 DEBUG:teuthology.parallel:result is None 2024-03-18T15:22:02.473 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-18T15:22:02.481 INFO:teuthology.task.internal:Creating test directory... 2024-03-18T15:22:02.481 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-18T15:22:02.483 DEBUG:teuthology.orchestra.run.smithi080:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-18T15:22:02.489 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-18T15:22:02.496 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-18T15:22:02.507 INFO:teuthology.task.internal:Creating archive directory... 2024-03-18T15:22:02.507 DEBUG:teuthology.orchestra.run.smithi005:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-18T15:22:02.533 DEBUG:teuthology.orchestra.run.smithi080:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-18T15:22:02.552 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-18T15:22:02.558 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-18T15:22:02.558 DEBUG:teuthology.orchestra.run.smithi005:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-18T15:22:02.588 DEBUG:teuthology.orchestra.run.smithi080:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-18T15:22:02.604 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-18T15:22:02.605 INFO:teuthology.orchestra.run.smithi080.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-18T15:22:02.612 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-18T15:22:02.614 INFO:teuthology.orchestra.run.smithi080.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-18T15:22:02.615 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-18T15:22:02.622 INFO:teuthology.task.internal:Configuring sudo... 2024-03-18T15:22:02.622 DEBUG:teuthology.orchestra.run.smithi005:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-18T15:22:02.657 DEBUG:teuthology.orchestra.run.smithi080:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-18T15:22:02.679 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-18T15:22:02.687 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-18T15:22:02.687 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-18T15:22:02.725 DEBUG:teuthology.orchestra.run.smithi080:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-18T15:22:02.731 DEBUG:teuthology.orchestra.run.smithi005:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-18T15:22:02.776 DEBUG:teuthology.orchestra.run.smithi005:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-18T15:22:02.824 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-03-18T15:22:02.824 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-18T15:22:02.886 DEBUG:teuthology.orchestra.run.smithi080:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-18T15:22:02.891 DEBUG:teuthology.orchestra.run.smithi080:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-18T15:22:02.938 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-03-18T15:22:02.938 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-18T15:22:02.999 DEBUG:teuthology.orchestra.run.smithi005:> sudo service rsyslog restart 2024-03-18T15:22:03.001 DEBUG:teuthology.orchestra.run.smithi080:> sudo service rsyslog restart 2024-03-18T15:22:03.071 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-18T15:22:03.081 INFO:teuthology.task.internal:Starting timer... 2024-03-18T15:22:03.081 INFO:teuthology.run_tasks:Running task pcp... 2024-03-18T15:22:03.091 INFO:teuthology.run_tasks:Running task selinux... 2024-03-18T15:22:03.099 DEBUG:teuthology.task.selinux:Excluding smithi005: OS 'ubuntu' does not support SELinux 2024-03-18T15:22:03.099 DEBUG:teuthology.task.selinux:Excluding smithi080: OS 'ubuntu' does not support SELinux 2024-03-18T15:22:03.100 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-18T15:22:03.100 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-03-18T15:22:03.100 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-18T15:22:03.100 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-18T15:22:03.109 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-18T15:22:03.125 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-18T15:22:03.127 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi005.front.sepia.ceph.com,smithi080.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-18T15:29:10.013 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi005.front.sepia.ceph.com'), Remote(name='ubuntu@smithi080.front.sepia.ceph.com')] 2024-03-18T15:29:10.015 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-03-18T15:29:10.016 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T15:29:10.111 DEBUG:teuthology.orchestra.run.smithi005:> true 2024-03-18T15:29:10.190 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-03-18T15:29:10.191 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi080.front.sepia.ceph.com' 2024-03-18T15:29:10.191 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-18T15:29:10.277 DEBUG:teuthology.orchestra.run.smithi080:> true 2024-03-18T15:29:10.355 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi080.front.sepia.ceph.com' 2024-03-18T15:29:10.355 INFO:teuthology.run_tasks:Running task clock... 2024-03-18T15:29:10.368 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-18T15:29:10.368 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-18T15:29:10.368 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-18T15:29:10.371 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-18T15:29:10.371 DEBUG:teuthology.orchestra.run.smithi080:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-18T15:29:10.400 INFO:teuthology.orchestra.run.smithi005.stdout:18 Mar 15:29:10 ntpd[17079]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-03-18T15:29:10.400 INFO:teuthology.orchestra.run.smithi005.stdout:18 Mar 15:29:10 ntpd[17079]: Command line: ntpd -gq 2024-03-18T15:29:10.400 INFO:teuthology.orchestra.run.smithi005.stdout:18 Mar 15:29:10 ntpd[17079]: ---------------------------------------------------- 2024-03-18T15:29:10.401 INFO:teuthology.orchestra.run.smithi005.stdout:18 Mar 15:29:10 ntpd[17079]: ntp-4 is maintained by Network Time Foundation, 2024-03-18T15:29:10.401 INFO:teuthology.orchestra.run.smithi005.stdout:18 Mar 15:29:10 ntpd[17079]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-03-18T15:29:10.401 INFO:teuthology.orchestra.run.smithi005.stdout:18 Mar 15:29:10 ntpd[17079]: corporation. Support and training for ntp-4 are 2024-03-18T15:29:10.401 INFO:teuthology.orchestra.run.smithi005.stdout:18 Mar 15:29:10 ntpd[17079]: available at https://www.nwtime.org/support 2024-03-18T15:29:10.401 INFO:teuthology.orchestra.run.smithi005.stdout:18 Mar 15:29:10 ntpd[17079]: ---------------------------------------------------- 2024-03-18T15:29:10.402 INFO:teuthology.orchestra.run.smithi005.stdout:18 Mar 15:29:10 ntpd[17079]: proto: precision = 0.062 usec (-24) 2024-03-18T15:29:10.402 INFO:teuthology.orchestra.run.smithi005.stdout:18 Mar 15:29:10 ntpd[17079]: basedate set to 2022-02-04 2024-03-18T15:29:10.402 INFO:teuthology.orchestra.run.smithi005.stdout:18 Mar 15:29:10 ntpd[17079]: gps base set to 2022-02-06 (week 2196) 2024-03-18T15:29:10.402 INFO:teuthology.orchestra.run.smithi005.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-18T15:29:10.403 INFO:teuthology.orchestra.run.smithi005.stdout:18 Mar 15:29:10 ntpd[17079]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-18T15:29:10.403 INFO:teuthology.orchestra.run.smithi005.stdout:18 Mar 15:29:10 ntpd[17079]: restrict ::: KOD does nothing without LIMITED. 2024-03-18T15:29:10.403 INFO:teuthology.orchestra.run.smithi005.stderr:restrict ::: KOD does nothing without LIMITED. 2024-03-18T15:29:10.404 INFO:teuthology.orchestra.run.smithi005.stdout:18 Mar 15:29:10 ntpd[17079]: Listen and drop on 0 v6wildcard [::]:123 2024-03-18T15:29:10.404 INFO:teuthology.orchestra.run.smithi005.stdout:18 Mar 15:29:10 ntpd[17079]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-03-18T15:29:10.404 INFO:teuthology.orchestra.run.smithi005.stdout:18 Mar 15:29:10 ntpd[17079]: Listen normally on 2 lo 127.0.0.1:123 2024-03-18T15:29:10.404 INFO:teuthology.orchestra.run.smithi005.stdout:18 Mar 15:29:10 ntpd[17079]: Listen normally on 3 ens1f0 172.21.15.5:123 2024-03-18T15:29:10.404 INFO:teuthology.orchestra.run.smithi005.stdout:18 Mar 15:29:10 ntpd[17079]: Listen normally on 4 lo [::1]:123 2024-03-18T15:29:10.404 INFO:teuthology.orchestra.run.smithi005.stdout:18 Mar 15:29:10 ntpd[17079]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:1442%4]:123 2024-03-18T15:29:10.404 INFO:teuthology.orchestra.run.smithi005.stdout:18 Mar 15:29:10 ntpd[17079]: Listening on routing socket on fd #22 for interface updates 2024-03-18T15:29:10.425 INFO:teuthology.orchestra.run.smithi080.stdout:18 Mar 15:29:10 ntpd[17168]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-03-18T15:29:10.425 INFO:teuthology.orchestra.run.smithi080.stdout:18 Mar 15:29:10 ntpd[17168]: Command line: ntpd -gq 2024-03-18T15:29:10.425 INFO:teuthology.orchestra.run.smithi080.stdout:18 Mar 15:29:10 ntpd[17168]: ---------------------------------------------------- 2024-03-18T15:29:10.426 INFO:teuthology.orchestra.run.smithi080.stdout:18 Mar 15:29:10 ntpd[17168]: ntp-4 is maintained by Network Time Foundation, 2024-03-18T15:29:10.426 INFO:teuthology.orchestra.run.smithi080.stdout:18 Mar 15:29:10 ntpd[17168]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-03-18T15:29:10.426 INFO:teuthology.orchestra.run.smithi080.stdout:18 Mar 15:29:10 ntpd[17168]: corporation. Support and training for ntp-4 are 2024-03-18T15:29:10.426 INFO:teuthology.orchestra.run.smithi080.stdout:18 Mar 15:29:10 ntpd[17168]: available at https://www.nwtime.org/support 2024-03-18T15:29:10.426 INFO:teuthology.orchestra.run.smithi080.stdout:18 Mar 15:29:10 ntpd[17168]: ---------------------------------------------------- 2024-03-18T15:29:10.426 INFO:teuthology.orchestra.run.smithi080.stdout:18 Mar 15:29:10 ntpd[17168]: proto: precision = 0.056 usec (-24) 2024-03-18T15:29:10.426 INFO:teuthology.orchestra.run.smithi080.stdout:18 Mar 15:29:10 ntpd[17168]: basedate set to 2022-02-04 2024-03-18T15:29:10.426 INFO:teuthology.orchestra.run.smithi080.stdout:18 Mar 15:29:10 ntpd[17168]: gps base set to 2022-02-06 (week 2196) 2024-03-18T15:29:10.426 INFO:teuthology.orchestra.run.smithi080.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-18T15:29:10.427 INFO:teuthology.orchestra.run.smithi080.stdout:18 Mar 15:29:10 ntpd[17168]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-18T15:29:10.427 INFO:teuthology.orchestra.run.smithi080.stderr:restrict ::: KOD does nothing without LIMITED. 2024-03-18T15:29:10.428 INFO:teuthology.orchestra.run.smithi080.stdout:18 Mar 15:29:10 ntpd[17168]: restrict ::: KOD does nothing without LIMITED. 2024-03-18T15:29:10.428 INFO:teuthology.orchestra.run.smithi080.stdout:18 Mar 15:29:10 ntpd[17168]: Listen and drop on 0 v6wildcard [::]:123 2024-03-18T15:29:10.428 INFO:teuthology.orchestra.run.smithi080.stdout:18 Mar 15:29:10 ntpd[17168]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-03-18T15:29:10.428 INFO:teuthology.orchestra.run.smithi080.stdout:18 Mar 15:29:10 ntpd[17168]: Listen normally on 2 lo 127.0.0.1:123 2024-03-18T15:29:10.428 INFO:teuthology.orchestra.run.smithi080.stdout:18 Mar 15:29:10 ntpd[17168]: Listen normally on 3 enp3s0f1 172.21.15.80:123 2024-03-18T15:29:10.428 INFO:teuthology.orchestra.run.smithi080.stdout:18 Mar 15:29:10 ntpd[17168]: Listen normally on 4 lo [::1]:123 2024-03-18T15:29:10.428 INFO:teuthology.orchestra.run.smithi080.stdout:18 Mar 15:29:10 ntpd[17168]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:cba9%5]:123 2024-03-18T15:29:10.429 INFO:teuthology.orchestra.run.smithi080.stdout:18 Mar 15:29:10 ntpd[17168]: Listening on routing socket on fd #22 for interface updates 2024-03-18T15:29:11.401 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:11 ntpd[17079]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-18T15:29:11.401 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:11 ntpd[17079]: can't open /var/log/ntpstats/rawstats.20240318: Permission denied 2024-03-18T15:29:11.401 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:11 ntpd[17079]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-18T15:29:11.401 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:11 ntpd[17079]: can't open /var/log/ntpstats/rawstats.20240318: Permission denied 2024-03-18T15:29:11.401 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:11 ntpd[17079]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-18T15:29:11.401 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:11 ntpd[17079]: can't open /var/log/ntpstats/peerstats.20240318: Permission denied 2024-03-18T15:29:11.425 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:11 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-18T15:29:11.426 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:11 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240318: Permission denied 2024-03-18T15:29:11.426 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:11 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-18T15:29:11.426 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:11 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240318: Permission denied 2024-03-18T15:29:11.426 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:11 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-18T15:29:11.426 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:11 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240318: Permission denied 2024-03-18T15:29:11.426 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:11 ntpd[17168]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-18T15:29:11.426 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:11 ntpd[17168]: can't open /var/log/ntpstats/peerstats.20240318: Permission denied 2024-03-18T15:29:12.401 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:12 ntpd[17079]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-18T15:29:12.401 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:12 ntpd[17079]: can't open /var/log/ntpstats/rawstats.20240318: Permission denied 2024-03-18T15:29:12.401 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:12 ntpd[17079]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-18T15:29:12.401 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:12 ntpd[17079]: can't open /var/log/ntpstats/rawstats.20240318: Permission denied 2024-03-18T15:29:12.401 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:12 ntpd[17079]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-18T15:29:12.401 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:12 ntpd[17079]: can't open /var/log/ntpstats/peerstats.20240318: Permission denied 2024-03-18T15:29:13.401 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:13 ntpd[17079]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-18T15:29:13.401 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:13 ntpd[17079]: can't open /var/log/ntpstats/rawstats.20240318: Permission denied 2024-03-18T15:29:13.401 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:13 ntpd[17079]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-18T15:29:13.401 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:13 ntpd[17079]: can't open /var/log/ntpstats/rawstats.20240318: Permission denied 2024-03-18T15:29:13.401 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:13 ntpd[17079]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-18T15:29:13.402 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:13 ntpd[17079]: can't open /var/log/ntpstats/peerstats.20240318: Permission denied 2024-03-18T15:29:13.426 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:13 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-18T15:29:13.426 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:13 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240318: Permission denied 2024-03-18T15:29:13.426 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:13 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-18T15:29:13.426 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:13 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240318: Permission denied 2024-03-18T15:29:13.426 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:13 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-18T15:29:13.426 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:13 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240318: Permission denied 2024-03-18T15:29:13.426 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:13 ntpd[17168]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-18T15:29:13.426 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:13 ntpd[17168]: can't open /var/log/ntpstats/peerstats.20240318: Permission denied 2024-03-18T15:29:13.426 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:13 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-18T15:29:13.427 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:13 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240318: Permission denied 2024-03-18T15:29:13.427 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:13 ntpd[17168]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-18T15:29:13.427 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:13 ntpd[17168]: can't open /var/log/ntpstats/peerstats.20240318: Permission denied 2024-03-18T15:29:14.401 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:14 ntpd[17079]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-18T15:29:14.402 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:14 ntpd[17079]: can't open /var/log/ntpstats/rawstats.20240318: Permission denied 2024-03-18T15:29:14.402 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:14 ntpd[17079]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-18T15:29:14.402 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:14 ntpd[17079]: can't open /var/log/ntpstats/rawstats.20240318: Permission denied 2024-03-18T15:29:14.402 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:14 ntpd[17079]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-18T15:29:14.402 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:14 ntpd[17079]: can't open /var/log/ntpstats/peerstats.20240318: Permission denied 2024-03-18T15:29:15.401 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:15 ntpd[17079]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-18T15:29:15.402 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:15 ntpd[17079]: can't open /var/log/ntpstats/rawstats.20240318: Permission denied 2024-03-18T15:29:15.402 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:15 ntpd[17079]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-18T15:29:15.402 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:15 ntpd[17079]: can't open /var/log/ntpstats/rawstats.20240318: Permission denied 2024-03-18T15:29:15.402 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:15 ntpd[17079]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-18T15:29:15.402 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:15 ntpd[17079]: can't open /var/log/ntpstats/rawstats.20240318: Permission denied 2024-03-18T15:29:15.402 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:15 ntpd[17079]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-18T15:29:15.402 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:15 ntpd[17079]: can't open /var/log/ntpstats/rawstats.20240318: Permission denied 2024-03-18T15:29:15.402 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:15 ntpd[17079]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-18T15:29:15.402 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:15 ntpd[17079]: can't open /var/log/ntpstats/peerstats.20240318: Permission denied 2024-03-18T15:29:15.402 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:15 ntpd[17079]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-18T15:29:15.402 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:15 ntpd[17079]: can't open /var/log/ntpstats/rawstats.20240318: Permission denied 2024-03-18T15:29:15.402 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:15 ntpd[17079]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-18T15:29:15.403 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:15 ntpd[17079]: can't open /var/log/ntpstats/peerstats.20240318: Permission denied 2024-03-18T15:29:15.426 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:15 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-18T15:29:15.426 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:15 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240318: Permission denied 2024-03-18T15:29:15.427 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:15 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-18T15:29:15.427 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:15 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240318: Permission denied 2024-03-18T15:29:15.427 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:15 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-18T15:29:15.427 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:15 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240318: Permission denied 2024-03-18T15:29:15.427 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:15 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-18T15:29:15.427 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:15 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240318: Permission denied 2024-03-18T15:29:15.427 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:15 ntpd[17168]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-18T15:29:15.427 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:15 ntpd[17168]: can't open /var/log/ntpstats/peerstats.20240318: Permission denied 2024-03-18T15:29:15.427 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:15 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-18T15:29:15.427 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:15 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240318: Permission denied 2024-03-18T15:29:15.427 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:15 ntpd[17168]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-18T15:29:15.427 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:15 ntpd[17168]: can't open /var/log/ntpstats/peerstats.20240318: Permission denied 2024-03-18T15:29:15.427 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:15 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-18T15:29:15.427 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:15 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240318: Permission denied 2024-03-18T15:29:15.428 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:15 ntpd[17168]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-18T15:29:15.428 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:15 ntpd[17168]: can't open /var/log/ntpstats/peerstats.20240318: Permission denied 2024-03-18T15:29:16.401 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:16 ntpd[17079]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-18T15:29:16.401 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:16 ntpd[17079]: can't open /var/log/ntpstats/rawstats.20240318: Permission denied 2024-03-18T15:29:16.401 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:16 ntpd[17079]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-18T15:29:16.401 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:16 ntpd[17079]: can't open /var/log/ntpstats/rawstats.20240318: Permission denied 2024-03-18T15:29:16.401 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:16 ntpd[17079]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-18T15:29:16.402 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:16 ntpd[17079]: can't open /var/log/ntpstats/peerstats.20240318: Permission denied 2024-03-18T15:29:17.401 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:17 ntpd[17079]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-18T15:29:17.402 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:17 ntpd[17079]: can't open /var/log/ntpstats/rawstats.20240318: Permission denied 2024-03-18T15:29:17.402 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:17 ntpd[17079]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-18T15:29:17.402 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:17 ntpd[17079]: can't open /var/log/ntpstats/rawstats.20240318: Permission denied 2024-03-18T15:29:17.402 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:17 ntpd[17079]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-18T15:29:17.402 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:17 ntpd[17079]: can't open /var/log/ntpstats/rawstats.20240318: Permission denied 2024-03-18T15:29:17.402 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:17 ntpd[17079]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-18T15:29:17.402 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:17 ntpd[17079]: can't open /var/log/ntpstats/peerstats.20240318: Permission denied 2024-03-18T15:29:17.402 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:17 ntpd[17079]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-18T15:29:17.402 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:17 ntpd[17079]: can't open /var/log/ntpstats/rawstats.20240318: Permission denied 2024-03-18T15:29:17.402 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:17 ntpd[17079]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-18T15:29:17.402 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:17 ntpd[17079]: can't open /var/log/ntpstats/peerstats.20240318: Permission denied 2024-03-18T15:29:17.403 INFO:teuthology.orchestra.run.smithi005.stdout:18 Mar 15:29:17 ntpd[17079]: ntpd: time slew -0.002699 s 2024-03-18T15:29:17.403 INFO:teuthology.orchestra.run.smithi005.stdout:ntpd: time slew -0.002699s 2024-03-18T15:29:17.403 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:17 ntpd[17079]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-03-18T15:29:17.403 INFO:teuthology.orchestra.run.smithi005.stderr:18 Mar 15:29:17 ntpd[17079]: can't open /var/log/ntpstats/loopstats.20240318: Permission denied 2024-03-18T15:29:17.426 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:17 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-18T15:29:17.426 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:17 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240318: Permission denied 2024-03-18T15:29:17.426 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:17 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-18T15:29:17.426 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:17 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240318: Permission denied 2024-03-18T15:29:17.426 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:17 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-18T15:29:17.426 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:17 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240318: Permission denied 2024-03-18T15:29:17.427 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:17 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-18T15:29:17.427 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:17 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240318: Permission denied 2024-03-18T15:29:17.427 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:17 ntpd[17168]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-18T15:29:17.427 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:17 ntpd[17168]: can't open /var/log/ntpstats/peerstats.20240318: Permission denied 2024-03-18T15:29:17.427 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:17 ntpd[17168]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-18T15:29:17.427 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:17 ntpd[17168]: can't open /var/log/ntpstats/rawstats.20240318: Permission denied 2024-03-18T15:29:17.427 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:17 ntpd[17168]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-18T15:29:17.427 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:17 ntpd[17168]: can't open /var/log/ntpstats/peerstats.20240318: Permission denied 2024-03-18T15:29:17.427 INFO:teuthology.orchestra.run.smithi080.stdout:18 Mar 15:29:17 ntpd[17168]: ntpd: time slew -0.000091 s 2024-03-18T15:29:17.427 INFO:teuthology.orchestra.run.smithi080.stdout:ntpd: time slew -0.000091s 2024-03-18T15:29:17.428 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:17 ntpd[17168]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-03-18T15:29:17.428 INFO:teuthology.orchestra.run.smithi080.stderr:18 Mar 15:29:17 ntpd[17168]: can't open /var/log/ntpstats/loopstats.20240318: Permission denied 2024-03-18T15:29:17.458 INFO:teuthology.orchestra.run.smithi005.stdout: remote refid st t when poll reach delay offset jitter 2024-03-18T15:29:17.458 INFO:teuthology.orchestra.run.smithi005.stdout:============================================================================== 2024-03-18T15:29:17.458 INFO:teuthology.orchestra.run.smithi005.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-18T15:29:17.458 INFO:teuthology.orchestra.run.smithi005.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-18T15:29:17.458 INFO:teuthology.orchestra.run.smithi005.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-18T15:29:17.459 INFO:teuthology.orchestra.run.smithi005.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-18T15:29:17.555 INFO:teuthology.orchestra.run.smithi080.stdout: remote refid st t when poll reach delay offset jitter 2024-03-18T15:29:17.556 INFO:teuthology.orchestra.run.smithi080.stdout:============================================================================== 2024-03-18T15:29:17.556 INFO:teuthology.orchestra.run.smithi080.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-18T15:29:17.556 INFO:teuthology.orchestra.run.smithi080.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-18T15:29:17.556 INFO:teuthology.orchestra.run.smithi080.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-18T15:29:17.556 INFO:teuthology.orchestra.run.smithi080.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-18T15:29:17.556 INFO:teuthology.run_tasks:Running task install... 2024-03-18T15:29:17.565 DEBUG:teuthology.task.install:project ceph 2024-03-18T15:29:17.565 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af'}} 2024-03-18T15:29:17.565 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af'} 2024-03-18T15:29:17.565 INFO:teuthology.task.install:Using flavor: default 2024-03-18T15:29:17.571 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-18T15:29:17.571 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-18T15:29:17.572 DEBUG:teuthology.orchestra.run.smithi005:> sudo apt-key list | grep Ceph 2024-03-18T15:29:17.572 DEBUG:teuthology.orchestra.run.smithi080:> sudo apt-key list | grep Ceph 2024-03-18T15:29:17.639 INFO:teuthology.orchestra.run.smithi005.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-03-18T15:29:17.657 INFO:teuthology.orchestra.run.smithi080.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-03-18T15:29:17.670 INFO:teuthology.orchestra.run.smithi005.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-03-18T15:29:17.670 INFO:teuthology.orchestra.run.smithi005.stdout:uid [ unknown] Ceph.com (release key) 2024-03-18T15:29:17.671 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-03-18T15:29:17.671 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-03-18T15:29:17.671 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-18T15:29:17.687 INFO:teuthology.orchestra.run.smithi080.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-03-18T15:29:17.687 INFO:teuthology.orchestra.run.smithi080.stdout:uid [ unknown] Ceph.com (release key) 2024-03-18T15:29:17.688 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-03-18T15:29:17.688 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-03-18T15:29:17.688 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-18T15:29:17.816 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default/ 2024-03-18T15:29:17.816 INFO:teuthology.task.install.deb:Package version is 19.0.0-2162-g3d78f5ba-1jammy 2024-03-18T15:29:17.827 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default/ 2024-03-18T15:29:17.827 INFO:teuthology.task.install.deb:Package version is 19.0.0-2162-g3d78f5ba-1jammy 2024-03-18T15:29:17.930 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-03-18T15:29:17.930 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-03-18T15:29:17.936 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-03-18T15:29:17.936 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-03-18T15:29:17.946 DEBUG:teuthology.orchestra.run.smithi005:> sudo apt-get update 2024-03-18T15:29:17.951 DEBUG:teuthology.orchestra.run.smithi080:> sudo apt-get update 2024-03-18T15:29:18.152 INFO:teuthology.orchestra.run.smithi005.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-18T15:29:18.153 INFO:teuthology.orchestra.run.smithi080.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-18T15:29:18.167 INFO:teuthology.orchestra.run.smithi080.stdout:Ign:2 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy InRelease 2024-03-18T15:29:18.170 INFO:teuthology.orchestra.run.smithi005.stdout:Ign:2 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy InRelease 2024-03-18T15:29:18.178 INFO:teuthology.orchestra.run.smithi080.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-18T15:29:18.179 INFO:teuthology.orchestra.run.smithi005.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-18T15:29:18.188 INFO:teuthology.orchestra.run.smithi080.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-03-18T15:29:18.194 INFO:teuthology.orchestra.run.smithi005.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-03-18T15:29:18.208 INFO:teuthology.orchestra.run.smithi080.stdout:Hit:5 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-18T15:29:18.211 INFO:teuthology.orchestra.run.smithi080.stdout:Ign:6 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy Release.gpg 2024-03-18T15:29:18.211 INFO:teuthology.orchestra.run.smithi005.stdout:Hit:5 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-18T15:29:18.213 INFO:teuthology.orchestra.run.smithi005.stdout:Ign:6 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy Release.gpg 2024-03-18T15:29:18.236 INFO:teuthology.orchestra.run.smithi080.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-03-18T15:29:18.236 INFO:teuthology.orchestra.run.smithi080.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,400 B] 2024-03-18T15:29:18.237 INFO:teuthology.orchestra.run.smithi005.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-03-18T15:29:18.237 INFO:teuthology.orchestra.run.smithi005.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-03-18T15:29:18.264 INFO:teuthology.orchestra.run.smithi080.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-03-18T15:29:18.265 INFO:teuthology.orchestra.run.smithi005.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,400 B] 2024-03-18T15:29:18.672 INFO:teuthology.orchestra.run.smithi080.stdout:Fetched 139 kB in 1s (258 kB/s) 2024-03-18T15:29:18.680 INFO:teuthology.orchestra.run.smithi005.stdout:Fetched 139 kB in 1s (256 kB/s) 2024-03-18T15:29:19.650 INFO:teuthology.orchestra.run.smithi080.stdout:Reading package lists... 2024-03-18T15:29:19.655 INFO:teuthology.orchestra.run.smithi005.stdout:Reading package lists... 2024-03-18T15:29:19.665 DEBUG:teuthology.orchestra.run.smithi080:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-2162-g3d78f5ba-1jammy cephadm=19.0.0-2162-g3d78f5ba-1jammy ceph-mds=19.0.0-2162-g3d78f5ba-1jammy ceph-mgr=19.0.0-2162-g3d78f5ba-1jammy ceph-common=19.0.0-2162-g3d78f5ba-1jammy ceph-fuse=19.0.0-2162-g3d78f5ba-1jammy ceph-test=19.0.0-2162-g3d78f5ba-1jammy ceph-volume=19.0.0-2162-g3d78f5ba-1jammy radosgw=19.0.0-2162-g3d78f5ba-1jammy python3-rados=19.0.0-2162-g3d78f5ba-1jammy python3-rgw=19.0.0-2162-g3d78f5ba-1jammy python3-cephfs=19.0.0-2162-g3d78f5ba-1jammy python3-rbd=19.0.0-2162-g3d78f5ba-1jammy libcephfs2=19.0.0-2162-g3d78f5ba-1jammy libcephfs-dev=19.0.0-2162-g3d78f5ba-1jammy librados2=19.0.0-2162-g3d78f5ba-1jammy librbd1=19.0.0-2162-g3d78f5ba-1jammy rbd-fuse=19.0.0-2162-g3d78f5ba-1jammy python3-cephfs=19.0.0-2162-g3d78f5ba-1jammy cephfs-shell=19.0.0-2162-g3d78f5ba-1jammy cephfs-top=19.0.0-2162-g3d78f5ba-1jammy cephfs-mirror=19.0.0-2162-g3d78f5ba-1jammy 2024-03-18T15:29:19.671 DEBUG:teuthology.orchestra.run.smithi005:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-2162-g3d78f5ba-1jammy cephadm=19.0.0-2162-g3d78f5ba-1jammy ceph-mds=19.0.0-2162-g3d78f5ba-1jammy ceph-mgr=19.0.0-2162-g3d78f5ba-1jammy ceph-common=19.0.0-2162-g3d78f5ba-1jammy ceph-fuse=19.0.0-2162-g3d78f5ba-1jammy ceph-test=19.0.0-2162-g3d78f5ba-1jammy ceph-volume=19.0.0-2162-g3d78f5ba-1jammy radosgw=19.0.0-2162-g3d78f5ba-1jammy python3-rados=19.0.0-2162-g3d78f5ba-1jammy python3-rgw=19.0.0-2162-g3d78f5ba-1jammy python3-cephfs=19.0.0-2162-g3d78f5ba-1jammy python3-rbd=19.0.0-2162-g3d78f5ba-1jammy libcephfs2=19.0.0-2162-g3d78f5ba-1jammy libcephfs-dev=19.0.0-2162-g3d78f5ba-1jammy librados2=19.0.0-2162-g3d78f5ba-1jammy librbd1=19.0.0-2162-g3d78f5ba-1jammy rbd-fuse=19.0.0-2162-g3d78f5ba-1jammy python3-cephfs=19.0.0-2162-g3d78f5ba-1jammy cephfs-shell=19.0.0-2162-g3d78f5ba-1jammy cephfs-top=19.0.0-2162-g3d78f5ba-1jammy cephfs-mirror=19.0.0-2162-g3d78f5ba-1jammy 2024-03-18T15:29:19.716 INFO:teuthology.orchestra.run.smithi080.stdout:Reading package lists... 2024-03-18T15:29:19.736 INFO:teuthology.orchestra.run.smithi005.stdout:Reading package lists... 2024-03-18T15:29:19.888 INFO:teuthology.orchestra.run.smithi080.stdout:Building dependency tree... 2024-03-18T15:29:19.889 INFO:teuthology.orchestra.run.smithi080.stdout:Reading state information... 2024-03-18T15:29:19.936 INFO:teuthology.orchestra.run.smithi005.stdout:Building dependency tree... 2024-03-18T15:29:19.937 INFO:teuthology.orchestra.run.smithi005.stdout:Reading state information... 2024-03-18T15:29:20.050 INFO:teuthology.orchestra.run.smithi080.stdout:The following packages were automatically installed and are no longer required: 2024-03-18T15:29:20.050 INFO:teuthology.orchestra.run.smithi080.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-18T15:29:20.050 INFO:teuthology.orchestra.run.smithi080.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-18T15:29:20.050 INFO:teuthology.orchestra.run.smithi080.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-18T15:29:20.050 INFO:teuthology.orchestra.run.smithi080.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-18T15:29:20.051 INFO:teuthology.orchestra.run.smithi080.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-18T15:29:20.051 INFO:teuthology.orchestra.run.smithi080.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-18T15:29:20.051 INFO:teuthology.orchestra.run.smithi080.stdout:The following additional packages will be installed: 2024-03-18T15:29:20.051 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-03-18T15:29:20.051 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-03-18T15:29:20.052 INFO:teuthology.orchestra.run.smithi080.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-03-18T15:29:20.052 INFO:teuthology.orchestra.run.smithi080.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-03-18T15:29:20.053 INFO:teuthology.orchestra.run.smithi080.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-03-18T15:29:20.053 INFO:teuthology.orchestra.run.smithi080.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-03-18T15:29:20.053 INFO:teuthology.orchestra.run.smithi080.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-03-18T15:29:20.053 INFO:teuthology.orchestra.run.smithi080.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-03-18T15:29:20.053 INFO:teuthology.orchestra.run.smithi080.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-03-18T15:29:20.053 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-03-18T15:29:20.053 INFO:teuthology.orchestra.run.smithi080.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-03-18T15:29:20.053 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-03-18T15:29:20.053 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-03-18T15:29:20.054 INFO:teuthology.orchestra.run.smithi080.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-03-18T15:29:20.054 INFO:teuthology.orchestra.run.smithi080.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-03-18T15:29:20.054 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-03-18T15:29:20.054 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-03-18T15:29:20.054 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-03-18T15:29:20.054 INFO:teuthology.orchestra.run.smithi080.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-03-18T15:29:20.054 INFO:teuthology.orchestra.run.smithi080.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-03-18T15:29:20.054 INFO:teuthology.orchestra.run.smithi080.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-03-18T15:29:20.054 INFO:teuthology.orchestra.run.smithi080.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-03-18T15:29:20.054 INFO:teuthology.orchestra.run.smithi080.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-03-18T15:29:20.054 INFO:teuthology.orchestra.run.smithi080.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-03-18T15:29:20.055 INFO:teuthology.orchestra.run.smithi080.stdout:Suggested packages: 2024-03-18T15:29:20.055 INFO:teuthology.orchestra.run.smithi080.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-03-18T15:29:20.055 INFO:teuthology.orchestra.run.smithi080.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-03-18T15:29:20.055 INFO:teuthology.orchestra.run.smithi080.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-03-18T15:29:20.055 INFO:teuthology.orchestra.run.smithi080.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-03-18T15:29:20.055 INFO:teuthology.orchestra.run.smithi080.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-03-18T15:29:20.055 INFO:teuthology.orchestra.run.smithi080.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-03-18T15:29:20.055 INFO:teuthology.orchestra.run.smithi080.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-03-18T15:29:20.055 INFO:teuthology.orchestra.run.smithi080.stdout:Recommended packages: 2024-03-18T15:29:20.055 INFO:teuthology.orchestra.run.smithi080.stdout: btrfs-tools 2024-03-18T15:29:20.127 INFO:teuthology.orchestra.run.smithi005.stdout:The following packages were automatically installed and are no longer required: 2024-03-18T15:29:20.127 INFO:teuthology.orchestra.run.smithi005.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-18T15:29:20.128 INFO:teuthology.orchestra.run.smithi005.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-18T15:29:20.128 INFO:teuthology.orchestra.run.smithi005.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-18T15:29:20.128 INFO:teuthology.orchestra.run.smithi005.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-18T15:29:20.128 INFO:teuthology.orchestra.run.smithi005.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-18T15:29:20.128 INFO:teuthology.orchestra.run.smithi005.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-18T15:29:20.129 INFO:teuthology.orchestra.run.smithi005.stdout:The following additional packages will be installed: 2024-03-18T15:29:20.129 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-03-18T15:29:20.129 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-03-18T15:29:20.130 INFO:teuthology.orchestra.run.smithi005.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-03-18T15:29:20.130 INFO:teuthology.orchestra.run.smithi005.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-03-18T15:29:20.130 INFO:teuthology.orchestra.run.smithi005.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-03-18T15:29:20.131 INFO:teuthology.orchestra.run.smithi005.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-03-18T15:29:20.131 INFO:teuthology.orchestra.run.smithi005.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-03-18T15:29:20.131 INFO:teuthology.orchestra.run.smithi005.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-03-18T15:29:20.131 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-03-18T15:29:20.131 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-03-18T15:29:20.131 INFO:teuthology.orchestra.run.smithi005.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-03-18T15:29:20.131 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-03-18T15:29:20.131 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-03-18T15:29:20.131 INFO:teuthology.orchestra.run.smithi005.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-03-18T15:29:20.131 INFO:teuthology.orchestra.run.smithi005.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-03-18T15:29:20.131 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-03-18T15:29:20.132 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-03-18T15:29:20.132 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-03-18T15:29:20.132 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-03-18T15:29:20.132 INFO:teuthology.orchestra.run.smithi005.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-03-18T15:29:20.132 INFO:teuthology.orchestra.run.smithi005.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-03-18T15:29:20.132 INFO:teuthology.orchestra.run.smithi005.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-03-18T15:29:20.132 INFO:teuthology.orchestra.run.smithi005.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-03-18T15:29:20.132 INFO:teuthology.orchestra.run.smithi005.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-03-18T15:29:20.132 INFO:teuthology.orchestra.run.smithi005.stdout:Suggested packages: 2024-03-18T15:29:20.132 INFO:teuthology.orchestra.run.smithi005.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-03-18T15:29:20.133 INFO:teuthology.orchestra.run.smithi005.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-03-18T15:29:20.133 INFO:teuthology.orchestra.run.smithi005.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-03-18T15:29:20.133 INFO:teuthology.orchestra.run.smithi005.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-03-18T15:29:20.133 INFO:teuthology.orchestra.run.smithi005.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-03-18T15:29:20.133 INFO:teuthology.orchestra.run.smithi005.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-03-18T15:29:20.133 INFO:teuthology.orchestra.run.smithi005.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-03-18T15:29:20.133 INFO:teuthology.orchestra.run.smithi005.stdout:Recommended packages: 2024-03-18T15:29:20.133 INFO:teuthology.orchestra.run.smithi005.stdout: btrfs-tools 2024-03-18T15:29:20.157 INFO:teuthology.orchestra.run.smithi080.stdout:The following NEW packages will be installed: 2024-03-18T15:29:20.157 INFO:teuthology.orchestra.run.smithi080.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-03-18T15:29:20.157 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-03-18T15:29:20.157 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-03-18T15:29:20.157 INFO:teuthology.orchestra.run.smithi080.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-03-18T15:29:20.158 INFO:teuthology.orchestra.run.smithi080.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-03-18T15:29:20.159 INFO:teuthology.orchestra.run.smithi080.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-03-18T15:29:20.159 INFO:teuthology.orchestra.run.smithi080.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-03-18T15:29:20.159 INFO:teuthology.orchestra.run.smithi080.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-03-18T15:29:20.159 INFO:teuthology.orchestra.run.smithi080.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-03-18T15:29:20.159 INFO:teuthology.orchestra.run.smithi080.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-03-18T15:29:20.159 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-03-18T15:29:20.159 INFO:teuthology.orchestra.run.smithi080.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-03-18T15:29:20.159 INFO:teuthology.orchestra.run.smithi080.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-03-18T15:29:20.159 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-03-18T15:29:20.159 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-03-18T15:29:20.159 INFO:teuthology.orchestra.run.smithi080.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-03-18T15:29:20.159 INFO:teuthology.orchestra.run.smithi080.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-03-18T15:29:20.159 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-03-18T15:29:20.160 INFO:teuthology.orchestra.run.smithi080.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-03-18T15:29:20.160 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-03-18T15:29:20.160 INFO:teuthology.orchestra.run.smithi080.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-03-18T15:29:20.160 INFO:teuthology.orchestra.run.smithi080.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-03-18T15:29:20.160 INFO:teuthology.orchestra.run.smithi080.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-03-18T15:29:20.160 INFO:teuthology.orchestra.run.smithi080.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-03-18T15:29:20.160 INFO:teuthology.orchestra.run.smithi080.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-03-18T15:29:20.160 INFO:teuthology.orchestra.run.smithi080.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-03-18T15:29:20.160 INFO:teuthology.orchestra.run.smithi080.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-03-18T15:29:20.160 INFO:teuthology.orchestra.run.smithi080.stdout: socat unzip xmlstarlet zip 2024-03-18T15:29:20.245 INFO:teuthology.orchestra.run.smithi005.stdout:The following NEW packages will be installed: 2024-03-18T15:29:20.245 INFO:teuthology.orchestra.run.smithi005.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-03-18T15:29:20.245 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-03-18T15:29:20.245 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-03-18T15:29:20.245 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-03-18T15:29:20.245 INFO:teuthology.orchestra.run.smithi005.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-03-18T15:29:20.245 INFO:teuthology.orchestra.run.smithi005.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-03-18T15:29:20.245 INFO:teuthology.orchestra.run.smithi005.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-03-18T15:29:20.247 INFO:teuthology.orchestra.run.smithi005.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-03-18T15:29:20.247 INFO:teuthology.orchestra.run.smithi005.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-03-18T15:29:20.247 INFO:teuthology.orchestra.run.smithi005.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-03-18T15:29:20.247 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-03-18T15:29:20.247 INFO:teuthology.orchestra.run.smithi005.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-03-18T15:29:20.247 INFO:teuthology.orchestra.run.smithi005.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-03-18T15:29:20.247 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-03-18T15:29:20.247 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-03-18T15:29:20.247 INFO:teuthology.orchestra.run.smithi005.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-03-18T15:29:20.247 INFO:teuthology.orchestra.run.smithi005.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-03-18T15:29:20.247 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-03-18T15:29:20.247 INFO:teuthology.orchestra.run.smithi005.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-03-18T15:29:20.247 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-03-18T15:29:20.248 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-03-18T15:29:20.248 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-03-18T15:29:20.248 INFO:teuthology.orchestra.run.smithi005.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-03-18T15:29:20.248 INFO:teuthology.orchestra.run.smithi005.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-03-18T15:29:20.248 INFO:teuthology.orchestra.run.smithi005.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-03-18T15:29:20.248 INFO:teuthology.orchestra.run.smithi005.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-03-18T15:29:20.248 INFO:teuthology.orchestra.run.smithi005.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-03-18T15:29:20.248 INFO:teuthology.orchestra.run.smithi005.stdout: socat unzip xmlstarlet zip 2024-03-18T15:29:20.273 INFO:teuthology.orchestra.run.smithi080.stdout:0 upgraded, 124 newly installed, 0 to remove and 285 not upgraded. 2024-03-18T15:29:20.274 INFO:teuthology.orchestra.run.smithi080.stdout:Need to get 178 MB of archives. 2024-03-18T15:29:20.274 INFO:teuthology.orchestra.run.smithi080.stdout:After this operation, 739 MB of additional disk space will be used. 2024-03-18T15:29:20.274 INFO:teuthology.orchestra.run.smithi080.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-03-18T15:29:20.321 INFO:teuthology.orchestra.run.smithi080.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-03-18T15:29:20.345 INFO:teuthology.orchestra.run.smithi080.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-03-18T15:29:20.390 INFO:teuthology.orchestra.run.smithi005.stdout:0 upgraded, 124 newly installed, 0 to remove and 285 not upgraded. 2024-03-18T15:29:20.390 INFO:teuthology.orchestra.run.smithi005.stdout:Need to get 178 MB of archives. 2024-03-18T15:29:20.390 INFO:teuthology.orchestra.run.smithi005.stdout:After this operation, 739 MB of additional disk space will be used. 2024-03-18T15:29:20.390 INFO:teuthology.orchestra.run.smithi005.stdout:Get:1 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-2162-g3d78f5ba-1jammy [3,554 kB] 2024-03-18T15:29:20.437 INFO:teuthology.orchestra.run.smithi005.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-03-18T15:29:20.459 INFO:teuthology.orchestra.run.smithi080.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-03-18T15:29:20.511 INFO:teuthology.orchestra.run.smithi080.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-2162-g3d78f5ba-1jammy [3,554 kB] 2024-03-18T15:29:20.516 INFO:teuthology.orchestra.run.smithi080.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-03-18T15:29:20.518 INFO:teuthology.orchestra.run.smithi005.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-03-18T15:29:20.523 INFO:teuthology.orchestra.run.smithi080.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-03-18T15:29:20.558 INFO:teuthology.orchestra.run.smithi080.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-03-18T15:29:20.564 INFO:teuthology.orchestra.run.smithi005.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-03-18T15:29:20.675 INFO:teuthology.orchestra.run.smithi080.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-03-18T15:29:20.682 INFO:teuthology.orchestra.run.smithi080.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-03-18T15:29:20.703 INFO:teuthology.orchestra.run.smithi080.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-03-18T15:29:20.726 INFO:teuthology.orchestra.run.smithi005.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-2162-g3d78f5ba-1jammy [3,225 kB] 2024-03-18T15:29:20.763 INFO:teuthology.orchestra.run.smithi080.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-03-18T15:29:20.764 INFO:teuthology.orchestra.run.smithi080.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-03-18T15:29:20.766 INFO:teuthology.orchestra.run.smithi005.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-03-18T15:29:20.779 INFO:teuthology.orchestra.run.smithi080.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-03-18T15:29:20.783 INFO:teuthology.orchestra.run.smithi080.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-03-18T15:29:20.787 INFO:teuthology.orchestra.run.smithi080.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-03-18T15:29:20.789 INFO:teuthology.orchestra.run.smithi080.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-03-18T15:29:20.790 INFO:teuthology.orchestra.run.smithi080.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-03-18T15:29:20.794 INFO:teuthology.orchestra.run.smithi080.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-03-18T15:29:20.798 INFO:teuthology.orchestra.run.smithi080.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-03-18T15:29:20.812 INFO:teuthology.orchestra.run.smithi080.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-03-18T15:29:20.813 INFO:teuthology.orchestra.run.smithi080.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-03-18T15:29:20.850 INFO:teuthology.orchestra.run.smithi080.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-03-18T15:29:20.850 INFO:teuthology.orchestra.run.smithi080.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-03-18T15:29:20.850 INFO:teuthology.orchestra.run.smithi080.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-03-18T15:29:20.851 INFO:teuthology.orchestra.run.smithi080.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-03-18T15:29:20.851 INFO:teuthology.orchestra.run.smithi080.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-03-18T15:29:20.852 INFO:teuthology.orchestra.run.smithi080.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-03-18T15:29:20.853 INFO:teuthology.orchestra.run.smithi080.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-03-18T15:29:20.853 INFO:teuthology.orchestra.run.smithi080.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-03-18T15:29:20.864 INFO:teuthology.orchestra.run.smithi005.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-03-18T15:29:20.875 INFO:teuthology.orchestra.run.smithi005.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-03-18T15:29:20.877 INFO:teuthology.orchestra.run.smithi005.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-2162-g3d78f5ba-1jammy [712 kB] 2024-03-18T15:29:20.885 INFO:teuthology.orchestra.run.smithi080.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-03-18T15:29:20.886 INFO:teuthology.orchestra.run.smithi080.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-03-18T15:29:20.900 INFO:teuthology.orchestra.run.smithi005.stdout:Get:10 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-2162-g3d78f5ba-1jammy [357 kB] 2024-03-18T15:29:20.917 INFO:teuthology.orchestra.run.smithi005.stdout:Get:11 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-2162-g3d78f5ba-1jammy [32.8 kB] 2024-03-18T15:29:20.918 INFO:teuthology.orchestra.run.smithi005.stdout:Get:12 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-2162-g3d78f5ba-1jammy [180 kB] 2024-03-18T15:29:20.921 INFO:teuthology.orchestra.run.smithi080.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-03-18T15:29:20.922 INFO:teuthology.orchestra.run.smithi080.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-03-18T15:29:20.922 INFO:teuthology.orchestra.run.smithi005.stdout:Get:13 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-2162-g3d78f5ba-1jammy [64.6 kB] 2024-03-18T15:29:20.923 INFO:teuthology.orchestra.run.smithi080.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-03-18T15:29:20.924 INFO:teuthology.orchestra.run.smithi005.stdout:Get:14 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-2162-g3d78f5ba-1jammy [330 kB] 2024-03-18T15:29:20.924 INFO:teuthology.orchestra.run.smithi080.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-03-18T15:29:20.925 INFO:teuthology.orchestra.run.smithi080.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-03-18T15:29:20.927 INFO:teuthology.orchestra.run.smithi005.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-03-18T15:29:20.927 INFO:teuthology.orchestra.run.smithi080.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-03-18T15:29:20.929 INFO:teuthology.orchestra.run.smithi080.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-03-18T15:29:20.930 INFO:teuthology.orchestra.run.smithi080.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-03-18T15:29:20.938 INFO:teuthology.orchestra.run.smithi005.stdout:Get:16 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-2162-g3d78f5ba-1jammy [6,653 kB] 2024-03-18T15:29:20.959 INFO:teuthology.orchestra.run.smithi080.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-03-18T15:29:20.959 INFO:teuthology.orchestra.run.smithi080.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-03-18T15:29:20.979 INFO:teuthology.orchestra.run.smithi080.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-03-18T15:29:20.980 INFO:teuthology.orchestra.run.smithi080.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-03-18T15:29:20.982 INFO:teuthology.orchestra.run.smithi080.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-03-18T15:29:20.986 INFO:teuthology.orchestra.run.smithi080.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-03-18T15:29:20.987 INFO:teuthology.orchestra.run.smithi080.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-03-18T15:29:20.990 INFO:teuthology.orchestra.run.smithi080.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-03-18T15:29:20.992 INFO:teuthology.orchestra.run.smithi080.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-03-18T15:29:21.000 INFO:teuthology.orchestra.run.smithi080.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-03-18T15:29:21.035 INFO:teuthology.orchestra.run.smithi080.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-03-18T15:29:21.036 INFO:teuthology.orchestra.run.smithi080.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-03-18T15:29:21.040 INFO:teuthology.orchestra.run.smithi080.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-03-18T15:29:21.101 INFO:teuthology.orchestra.run.smithi005.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-03-18T15:29:21.111 INFO:teuthology.orchestra.run.smithi005.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-03-18T15:29:21.137 INFO:teuthology.orchestra.run.smithi005.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-03-18T15:29:21.145 INFO:teuthology.orchestra.run.smithi005.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-03-18T15:29:21.169 INFO:teuthology.orchestra.run.smithi005.stdout:Get:21 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-2162-g3d78f5ba-1jammy [112 kB] 2024-03-18T15:29:21.172 INFO:teuthology.orchestra.run.smithi005.stdout:Get:22 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-2162-g3d78f5ba-1jammy [466 kB] 2024-03-18T15:29:21.187 INFO:teuthology.orchestra.run.smithi005.stdout:Get:23 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-2162-g3d78f5ba-1jammy [25.3 MB] 2024-03-18T15:29:21.198 INFO:teuthology.orchestra.run.smithi080.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-03-18T15:29:21.199 INFO:teuthology.orchestra.run.smithi080.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-03-18T15:29:21.201 INFO:teuthology.orchestra.run.smithi080.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-03-18T15:29:21.202 INFO:teuthology.orchestra.run.smithi080.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-03-18T15:29:21.204 INFO:teuthology.orchestra.run.smithi080.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-03-18T15:29:21.229 INFO:teuthology.orchestra.run.smithi080.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-03-18T15:29:21.231 INFO:teuthology.orchestra.run.smithi080.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-03-18T15:29:21.231 INFO:teuthology.orchestra.run.smithi080.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-03-18T15:29:21.268 INFO:teuthology.orchestra.run.smithi080.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-03-18T15:29:21.268 INFO:teuthology.orchestra.run.smithi080.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-03-18T15:29:21.288 INFO:teuthology.orchestra.run.smithi080.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-03-18T15:29:21.288 INFO:teuthology.orchestra.run.smithi080.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-03-18T15:29:21.289 INFO:teuthology.orchestra.run.smithi080.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-03-18T15:29:21.290 INFO:teuthology.orchestra.run.smithi080.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-03-18T15:29:21.290 INFO:teuthology.orchestra.run.smithi080.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-03-18T15:29:21.291 INFO:teuthology.orchestra.run.smithi080.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-03-18T15:29:21.296 INFO:teuthology.orchestra.run.smithi080.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-03-18T15:29:21.311 INFO:teuthology.orchestra.run.smithi005.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-03-18T15:29:21.318 INFO:teuthology.orchestra.run.smithi080.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-03-18T15:29:21.319 INFO:teuthology.orchestra.run.smithi080.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-03-18T15:29:21.321 INFO:teuthology.orchestra.run.smithi080.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-03-18T15:29:21.346 INFO:teuthology.orchestra.run.smithi080.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-03-18T15:29:21.351 INFO:teuthology.orchestra.run.smithi080.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-03-18T15:29:21.351 INFO:teuthology.orchestra.run.smithi080.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-03-18T15:29:21.353 INFO:teuthology.orchestra.run.smithi080.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-03-18T15:29:21.354 INFO:teuthology.orchestra.run.smithi080.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-03-18T15:29:21.362 INFO:teuthology.orchestra.run.smithi080.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-03-18T15:29:21.363 INFO:teuthology.orchestra.run.smithi080.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-03-18T15:29:21.381 INFO:teuthology.orchestra.run.smithi080.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-03-18T15:29:21.382 INFO:teuthology.orchestra.run.smithi080.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-03-18T15:29:21.383 INFO:teuthology.orchestra.run.smithi080.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-03-18T15:29:21.410 INFO:teuthology.orchestra.run.smithi080.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-03-18T15:29:21.414 INFO:teuthology.orchestra.run.smithi080.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-03-18T15:29:21.415 INFO:teuthology.orchestra.run.smithi080.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-03-18T15:29:21.430 INFO:teuthology.orchestra.run.smithi080.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-03-18T15:29:21.430 INFO:teuthology.orchestra.run.smithi080.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-03-18T15:29:21.431 INFO:teuthology.orchestra.run.smithi080.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-03-18T15:29:21.435 INFO:teuthology.orchestra.run.smithi080.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-03-18T15:29:21.446 INFO:teuthology.orchestra.run.smithi080.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-03-18T15:29:21.884 INFO:teuthology.orchestra.run.smithi005.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-03-18T15:29:21.899 INFO:teuthology.orchestra.run.smithi005.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-03-18T15:29:21.915 INFO:teuthology.orchestra.run.smithi005.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-03-18T15:29:21.924 INFO:teuthology.orchestra.run.smithi005.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-03-18T15:29:21.924 INFO:teuthology.orchestra.run.smithi005.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-03-18T15:29:21.940 INFO:teuthology.orchestra.run.smithi005.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-03-18T15:29:21.978 INFO:teuthology.orchestra.run.smithi005.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-03-18T15:29:21.991 INFO:teuthology.orchestra.run.smithi005.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-03-18T15:29:21.995 INFO:teuthology.orchestra.run.smithi005.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-03-18T15:29:21.996 INFO:teuthology.orchestra.run.smithi005.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-03-18T15:29:22.126 INFO:teuthology.orchestra.run.smithi005.stdout:Get:35 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-2162-g3d78f5ba-1jammy [5,085 kB] 2024-03-18T15:29:22.167 INFO:teuthology.orchestra.run.smithi005.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-03-18T15:29:22.229 INFO:teuthology.orchestra.run.smithi005.stdout:Get:37 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-2162-g3d78f5ba-1jammy [244 kB] 2024-03-18T15:29:22.236 INFO:teuthology.orchestra.run.smithi005.stdout:Get:38 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-2162-g3d78f5ba-1jammy [124 kB] 2024-03-18T15:29:22.240 INFO:teuthology.orchestra.run.smithi005.stdout:Get:39 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-2162-g3d78f5ba-1jammy [1,526 kB] 2024-03-18T15:29:22.252 INFO:teuthology.orchestra.run.smithi005.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-03-18T15:29:22.272 INFO:teuthology.orchestra.run.smithi005.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-03-18T15:29:22.332 INFO:teuthology.orchestra.run.smithi005.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-03-18T15:29:22.364 INFO:teuthology.orchestra.run.smithi005.stdout:Get:43 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-2162-g3d78f5ba-1jammy [6,255 kB] 2024-03-18T15:29:22.372 INFO:teuthology.orchestra.run.smithi005.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-03-18T15:29:22.395 INFO:teuthology.orchestra.run.smithi005.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-03-18T15:29:22.422 INFO:teuthology.orchestra.run.smithi005.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-03-18T15:29:22.512 INFO:teuthology.orchestra.run.smithi005.stdout:Get:47 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-2162-g3d78f5ba-1jammy [22.8 MB] 2024-03-18T15:29:22.606 INFO:teuthology.orchestra.run.smithi005.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-03-18T15:29:22.622 INFO:teuthology.orchestra.run.smithi005.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-03-18T15:29:22.629 INFO:teuthology.orchestra.run.smithi005.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-03-18T15:29:22.634 INFO:teuthology.orchestra.run.smithi005.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-03-18T15:29:22.820 INFO:teuthology.orchestra.run.smithi005.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-03-18T15:29:22.906 INFO:teuthology.orchestra.run.smithi005.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-03-18T15:29:22.917 INFO:teuthology.orchestra.run.smithi005.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-03-18T15:29:23.097 INFO:teuthology.orchestra.run.smithi005.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-03-18T15:29:23.142 INFO:teuthology.orchestra.run.smithi005.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-03-18T15:29:23.148 INFO:teuthology.orchestra.run.smithi080.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-2162-g3d78f5ba-1jammy [3,225 kB] 2024-03-18T15:29:23.155 INFO:teuthology.orchestra.run.smithi005.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-03-18T15:29:23.347 INFO:teuthology.orchestra.run.smithi005.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-03-18T15:29:23.369 INFO:teuthology.orchestra.run.smithi005.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-03-18T15:29:23.422 INFO:teuthology.orchestra.run.smithi005.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-03-18T15:29:23.442 INFO:teuthology.orchestra.run.smithi005.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-03-18T15:29:23.467 INFO:teuthology.orchestra.run.smithi005.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-03-18T15:29:23.467 INFO:teuthology.orchestra.run.smithi005.stdout:Get:63 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-2162-g3d78f5ba-1jammy [14.1 kB] 2024-03-18T15:29:23.593 INFO:teuthology.orchestra.run.smithi005.stdout:Get:64 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-2162-g3d78f5ba-1jammy [870 kB] 2024-03-18T15:29:23.624 INFO:teuthology.orchestra.run.smithi005.stdout:Get:65 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-2162-g3d78f5ba-1jammy [2,434 kB] 2024-03-18T15:29:23.669 INFO:teuthology.orchestra.run.smithi005.stdout:Get:66 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-2162-g3d78f5ba-1jammy [777 kB] 2024-03-18T15:29:23.681 INFO:teuthology.orchestra.run.smithi005.stdout:Get:67 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-2162-g3d78f5ba-1jammy [144 kB] 2024-03-18T15:29:23.683 INFO:teuthology.orchestra.run.smithi005.stdout:Get:68 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-2162-g3d78f5ba-1jammy [1,936 kB] 2024-03-18T15:29:23.704 INFO:teuthology.orchestra.run.smithi005.stdout:Get:69 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-2162-g3d78f5ba-1jammy [8,625 kB] 2024-03-18T15:29:23.757 INFO:teuthology.orchestra.run.smithi005.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-03-18T15:29:23.985 INFO:teuthology.orchestra.run.smithi005.stdout:Get:71 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-2162-g3d78f5ba-1jammy [14.2 kB] 2024-03-18T15:29:23.986 INFO:teuthology.orchestra.run.smithi005.stdout:Get:72 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-2162-g3d78f5ba-1jammy [50.2 MB] 2024-03-18T15:29:24.010 INFO:teuthology.orchestra.run.smithi005.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-03-18T15:29:24.122 INFO:teuthology.orchestra.run.smithi005.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-03-18T15:29:24.166 INFO:teuthology.orchestra.run.smithi005.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-03-18T15:29:24.248 INFO:teuthology.orchestra.run.smithi005.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-03-18T15:29:24.261 INFO:teuthology.orchestra.run.smithi005.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-03-18T15:29:24.274 INFO:teuthology.orchestra.run.smithi005.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-03-18T15:29:24.313 INFO:teuthology.orchestra.run.smithi005.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-03-18T15:29:24.574 INFO:teuthology.orchestra.run.smithi005.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-03-18T15:29:24.575 INFO:teuthology.orchestra.run.smithi005.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-03-18T15:29:24.578 INFO:teuthology.orchestra.run.smithi005.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-03-18T15:29:24.742 INFO:teuthology.orchestra.run.smithi005.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-03-18T15:29:25.067 INFO:teuthology.orchestra.run.smithi005.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-03-18T15:29:25.477 INFO:teuthology.orchestra.run.smithi005.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-03-18T15:29:25.486 INFO:teuthology.orchestra.run.smithi005.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-03-18T15:29:25.487 INFO:teuthology.orchestra.run.smithi005.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-03-18T15:29:25.557 INFO:teuthology.orchestra.run.smithi005.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-03-18T15:29:25.558 INFO:teuthology.orchestra.run.smithi005.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-03-18T15:29:25.560 INFO:teuthology.orchestra.run.smithi005.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-03-18T15:29:25.561 INFO:teuthology.orchestra.run.smithi005.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-03-18T15:29:25.562 INFO:teuthology.orchestra.run.smithi005.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-03-18T15:29:25.727 INFO:teuthology.orchestra.run.smithi005.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-03-18T15:29:25.767 INFO:teuthology.orchestra.run.smithi080.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-2162-g3d78f5ba-1jammy [712 kB] 2024-03-18T15:29:25.860 INFO:teuthology.orchestra.run.smithi005.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-03-18T15:29:25.931 INFO:teuthology.orchestra.run.smithi005.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-03-18T15:29:25.932 INFO:teuthology.orchestra.run.smithi005.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-2162-g3d78f5ba-1jammy [121 kB] 2024-03-18T15:29:25.934 INFO:teuthology.orchestra.run.smithi005.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-2162-g3d78f5ba-1jammy [230 kB] 2024-03-18T15:29:25.939 INFO:teuthology.orchestra.run.smithi005.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-2162-g3d78f5ba-1jammy [34.0 kB] 2024-03-18T15:29:25.940 INFO:teuthology.orchestra.run.smithi005.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-2162-g3d78f5ba-1jammy [23.4 kB] 2024-03-18T15:29:25.940 INFO:teuthology.orchestra.run.smithi005.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-2162-g3d78f5ba-1jammy [39.0 kB] 2024-03-18T15:29:25.941 INFO:teuthology.orchestra.run.smithi005.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-2162-g3d78f5ba-1jammy [13.1 MB] 2024-03-18T15:29:26.152 INFO:teuthology.orchestra.run.smithi080.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-2162-g3d78f5ba-1jammy [357 kB] 2024-03-18T15:29:26.163 INFO:teuthology.orchestra.run.smithi005.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-03-18T15:29:26.206 INFO:teuthology.orchestra.run.smithi005.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-03-18T15:29:26.230 INFO:teuthology.orchestra.run.smithi005.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-03-18T15:29:26.239 INFO:teuthology.orchestra.run.smithi005.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-03-18T15:29:26.277 INFO:teuthology.orchestra.run.smithi005.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-2162-g3d78f5ba-1jammy [91.5 kB] 2024-03-18T15:29:26.282 INFO:teuthology.orchestra.run.smithi005.stdout:Get:107 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-03-18T15:29:26.307 INFO:teuthology.orchestra.run.smithi005.stdout:Get:108 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-03-18T15:29:26.310 INFO:teuthology.orchestra.run.smithi005.stdout:Get:109 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-03-18T15:29:26.316 INFO:teuthology.orchestra.run.smithi005.stdout:Get:110 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-03-18T15:29:26.346 INFO:teuthology.orchestra.run.smithi080.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-2162-g3d78f5ba-1jammy [32.8 kB] 2024-03-18T15:29:26.380 INFO:teuthology.orchestra.run.smithi080.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-2162-g3d78f5ba-1jammy [180 kB] 2024-03-18T15:29:26.435 INFO:teuthology.orchestra.run.smithi080.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-2162-g3d78f5ba-1jammy [64.6 kB] 2024-03-18T15:29:26.465 INFO:teuthology.orchestra.run.smithi080.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-2162-g3d78f5ba-1jammy [330 kB] 2024-03-18T15:29:26.482 INFO:teuthology.orchestra.run.smithi005.stdout:Get:111 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-03-18T15:29:26.563 INFO:teuthology.orchestra.run.smithi080.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-2162-g3d78f5ba-1jammy [6,653 kB] 2024-03-18T15:29:27.027 INFO:teuthology.orchestra.run.smithi005.stdout:Get:112 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-03-18T15:29:27.029 INFO:teuthology.orchestra.run.smithi005.stdout:Get:113 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-03-18T15:29:27.068 INFO:teuthology.orchestra.run.smithi005.stdout:Get:114 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-03-18T15:29:27.069 INFO:teuthology.orchestra.run.smithi005.stdout:Get:115 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-03-18T15:29:27.075 INFO:teuthology.orchestra.run.smithi005.stdout:Get:116 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-03-18T15:29:27.078 INFO:teuthology.orchestra.run.smithi005.stdout:Get:117 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-03-18T15:29:27.118 INFO:teuthology.orchestra.run.smithi005.stdout:Get:118 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-03-18T15:29:27.126 INFO:teuthology.orchestra.run.smithi005.stdout:Get:119 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-03-18T15:29:27.200 INFO:teuthology.orchestra.run.smithi005.stdout:Get:120 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-03-18T15:29:27.201 INFO:teuthology.orchestra.run.smithi005.stdout:Get:121 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-03-18T15:29:27.365 INFO:teuthology.orchestra.run.smithi005.stdout:Get:122 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-03-18T15:29:27.804 INFO:teuthology.orchestra.run.smithi005.stdout:Get:123 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-03-18T15:29:27.819 INFO:teuthology.orchestra.run.smithi005.stdout:Get:124 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-03-18T15:29:28.758 INFO:teuthology.orchestra.run.smithi005.stdout:Fetched 178 MB in 8s (22.7 MB/s) 2024-03-18T15:29:28.887 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-pyperclip. 2024-03-18T15:29:28.982 INFO:teuthology.orchestra.run.smithi005.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165997 files and directories currently installed.) 2024-03-18T15:29:28.988 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-03-18T15:29:29.027 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-03-18T15:29:29.175 INFO:teuthology.orchestra.run.smithi080.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-2162-g3d78f5ba-1jammy [112 kB] 2024-03-18T15:29:29.226 INFO:teuthology.orchestra.run.smithi080.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-2162-g3d78f5ba-1jammy [466 kB] 2024-03-18T15:29:29.321 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-wcwidth. 2024-03-18T15:29:29.348 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-03-18T15:29:29.388 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-18T15:29:29.477 INFO:teuthology.orchestra.run.smithi080.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-2162-g3d78f5ba-1jammy [25.3 MB] 2024-03-18T15:29:29.707 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-cmd2. 2024-03-18T15:29:29.723 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-03-18T15:29:29.765 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-03-18T15:29:30.127 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-03-18T15:29:30.154 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-03-18T15:29:30.235 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-18T15:29:30.555 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-03-18T15:29:30.582 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-03-18T15:29:30.613 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-18T15:29:30.941 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-03-18T15:29:30.968 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-03-18T15:29:31.008 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-18T15:29:31.411 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-03-18T15:29:31.438 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-18T15:29:31.486 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-18T15:29:31.898 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-03-18T15:29:31.925 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-18T15:29:31.964 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-18T15:29:32.317 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-03-18T15:29:32.344 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-18T15:29:32.384 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-18T15:29:32.679 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-03-18T15:29:32.694 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-03-18T15:29:32.737 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-18T15:29:33.065 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package librados2. 2024-03-18T15:29:33.091 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../010-librados2_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-18T15:29:33.131 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking librados2 (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:29:33.776 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package librbd1. 2024-03-18T15:29:33.802 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../011-librbd1_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-18T15:29:33.842 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking librbd1 (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:29:34.278 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package libcephfs2. 2024-03-18T15:29:34.294 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-18T15:29:34.328 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking libcephfs2 (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:29:34.589 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-rados. 2024-03-18T15:29:34.605 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../013-python3-rados_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-18T15:29:34.647 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-rados (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:29:34.992 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-03-18T15:29:35.018 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-2162-g3d78f5ba-1jammy_all.deb ... 2024-03-18T15:29:35.058 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-ceph-argparse (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:29:35.319 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-cephfs. 2024-03-18T15:29:35.335 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-18T15:29:35.377 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-cephfs (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:29:35.663 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-ceph-common. 2024-03-18T15:29:35.679 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-2162-g3d78f5ba-1jammy_all.deb ... 2024-03-18T15:29:35.713 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-ceph-common (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:29:36.016 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-prettytable. 2024-03-18T15:29:36.032 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-03-18T15:29:36.066 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-03-18T15:29:36.327 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-rbd. 2024-03-18T15:29:36.354 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-18T15:29:36.394 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-rbd (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:29:36.746 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-03-18T15:29:36.773 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-03-18T15:29:36.813 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-18T15:29:37.090 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-03-18T15:29:37.117 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-03-18T15:29:37.157 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-03-18T15:29:37.451 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-03-18T15:29:37.468 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-03-18T15:29:37.501 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-18T15:29:37.788 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package lua5.1. 2024-03-18T15:29:37.804 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-03-18T15:29:37.863 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-03-18T15:29:38.174 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package lua-any. 2024-03-18T15:29:38.201 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-03-18T15:29:38.241 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking lua-any (27ubuntu1) ... 2024-03-18T15:29:38.493 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package zip. 2024-03-18T15:29:38.510 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-03-18T15:29:38.552 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking zip (3.0-12build2) ... 2024-03-18T15:29:38.888 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package unzip. 2024-03-18T15:29:38.915 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-03-18T15:29:38.955 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-03-18T15:29:39.292 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package luarocks. 2024-03-18T15:29:39.308 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-03-18T15:29:39.350 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-03-18T15:29:39.733 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package librgw2. 2024-03-18T15:29:39.760 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../027-librgw2_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-18T15:29:39.799 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking librgw2 (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:29:40.335 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-rgw. 2024-03-18T15:29:40.351 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-18T15:29:40.393 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-rgw (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:29:40.688 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package liboath0:amd64. 2024-03-18T15:29:40.715 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-03-18T15:29:40.747 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-03-18T15:29:41.049 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package libradosstriper1. 2024-03-18T15:29:41.076 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-18T15:29:41.116 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking libradosstriper1 (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:29:41.577 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package ceph-common. 2024-03-18T15:29:41.604 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../031-ceph-common_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-18T15:29:41.643 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking ceph-common (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:29:42.932 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package ceph-base. 2024-03-18T15:29:42.959 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../032-ceph-base_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-18T15:29:43.008 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking ceph-base (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:29:43.338 INFO:teuthology.orchestra.run.smithi080.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-2162-g3d78f5ba-1jammy [5,085 kB] 2024-03-18T15:29:43.628 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-03-18T15:29:43.655 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-03-18T15:29:43.740 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-03-18T15:29:44.189 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-cheroot. 2024-03-18T15:29:44.216 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-03-18T15:29:44.256 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-18T15:29:44.575 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-03-18T15:29:44.602 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-03-18T15:29:44.642 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-03-18T15:29:44.670 INFO:teuthology.orchestra.run.smithi080.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-2162-g3d78f5ba-1jammy [244 kB] 2024-03-18T15:29:44.725 INFO:teuthology.orchestra.run.smithi080.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-2162-g3d78f5ba-1jammy [124 kB] 2024-03-18T15:29:44.750 INFO:teuthology.orchestra.run.smithi080.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-2162-g3d78f5ba-1jammy [1,526 kB] 2024-03-18T15:29:44.962 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-jaraco.text. 2024-03-18T15:29:44.989 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-03-18T15:29:45.036 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-03-18T15:29:45.087 INFO:teuthology.orchestra.run.smithi080.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-2162-g3d78f5ba-1jammy [6,255 kB] 2024-03-18T15:29:45.289 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-03-18T15:29:45.316 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-03-18T15:29:45.348 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-03-18T15:29:45.634 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-tempora. 2024-03-18T15:29:45.661 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-03-18T15:29:45.692 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-03-18T15:29:45.970 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-portend. 2024-03-18T15:29:45.997 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-03-18T15:29:46.036 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-03-18T15:29:46.339 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-03-18T15:29:46.366 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-03-18T15:29:46.406 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-03-18T15:29:46.452 INFO:teuthology.orchestra.run.smithi080.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-2162-g3d78f5ba-1jammy [22.8 MB] 2024-03-18T15:29:46.709 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-cherrypy3. 2024-03-18T15:29:46.735 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-03-18T15:29:46.775 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-03-18T15:29:47.129 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-natsort. 2024-03-18T15:29:47.156 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-03-18T15:29:47.195 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-03-18T15:29:47.506 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-logutils. 2024-03-18T15:29:47.533 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-03-18T15:29:47.573 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-03-18T15:29:47.867 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-markupsafe. 2024-03-18T15:29:47.894 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-03-18T15:29:47.934 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-03-18T15:29:48.187 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-mako. 2024-03-18T15:29:48.203 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-03-18T15:29:48.245 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-18T15:29:48.623 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-simplegeneric. 2024-03-18T15:29:48.650 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-03-18T15:29:48.690 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-03-18T15:29:48.993 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-singledispatch. 2024-03-18T15:29:49.020 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-03-18T15:29:49.059 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-03-18T15:29:49.312 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-webob. 2024-03-18T15:29:49.339 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-03-18T15:29:49.370 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-03-18T15:29:49.640 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-waitress. 2024-03-18T15:29:49.667 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-03-18T15:29:49.710 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-18T15:29:50.009 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-tempita. 2024-03-18T15:29:50.036 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-03-18T15:29:50.076 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-03-18T15:29:50.379 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-paste. 2024-03-18T15:29:50.406 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-03-18T15:29:50.445 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-03-18T15:29:50.757 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-03-18T15:29:50.784 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-03-18T15:29:50.823 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-03-18T15:29:51.109 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-pastedeploy. 2024-03-18T15:29:51.136 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-03-18T15:29:51.175 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-03-18T15:29:51.479 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-webtest. 2024-03-18T15:29:51.506 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-03-18T15:29:51.545 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-03-18T15:29:51.806 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-pecan. 2024-03-18T15:29:51.834 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-03-18T15:29:51.865 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-03-18T15:29:52.161 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-certifi. 2024-03-18T15:29:52.177 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-03-18T15:29:52.211 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-03-18T15:29:52.556 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-idna. 2024-03-18T15:29:52.583 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-03-18T15:29:52.622 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-idna (3.3-1) ... 2024-03-18T15:29:52.933 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-urllib3. 2024-03-18T15:29:52.961 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-03-18T15:29:53.000 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-18T15:29:53.328 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-requests. 2024-03-18T15:29:53.355 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-03-18T15:29:53.394 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-18T15:29:54.456 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-werkzeug. 2024-03-18T15:29:54.483 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-03-18T15:29:54.514 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-18T15:29:54.800 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-03-18T15:29:54.827 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-2162-g3d78f5ba-1jammy_all.deb ... 2024-03-18T15:29:54.867 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking ceph-mgr-modules-core (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:29:55.264 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-03-18T15:29:55.291 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-18T15:29:55.330 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:29:55.641 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-bcrypt. 2024-03-18T15:29:55.669 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-03-18T15:29:55.707 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-03-18T15:29:56.035 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-openssl. 2024-03-18T15:29:56.063 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-03-18T15:29:56.102 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-03-18T15:29:56.396 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package ceph-mgr. 2024-03-18T15:29:56.413 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-18T15:29:56.446 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking ceph-mgr (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:29:56.825 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package ceph-mon. 2024-03-18T15:29:56.852 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-18T15:29:56.891 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking ceph-mon (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:29:56.944 INFO:teuthology.orchestra.run.smithi080.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-2162-g3d78f5ba-1jammy [14.1 kB] 2024-03-18T15:29:56.945 INFO:teuthology.orchestra.run.smithi080.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-2162-g3d78f5ba-1jammy [870 kB] 2024-03-18T15:29:57.077 INFO:teuthology.orchestra.run.smithi080.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-2162-g3d78f5ba-1jammy [2,434 kB] 2024-03-18T15:29:57.494 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package ceph-osd. 2024-03-18T15:29:57.522 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-18T15:29:57.561 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking ceph-osd (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:29:57.648 INFO:teuthology.orchestra.run.smithi080.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-2162-g3d78f5ba-1jammy [777 kB] 2024-03-18T15:29:57.873 INFO:teuthology.orchestra.run.smithi080.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-2162-g3d78f5ba-1jammy [144 kB] 2024-03-18T15:29:57.935 INFO:teuthology.orchestra.run.smithi080.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-2162-g3d78f5ba-1jammy [1,936 kB] 2024-03-18T15:29:58.698 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package ceph. 2024-03-18T15:29:58.714 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../068-ceph_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-18T15:29:58.746 INFO:teuthology.orchestra.run.smithi080.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-2162-g3d78f5ba-1jammy [8,625 kB] 2024-03-18T15:29:58.756 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking ceph (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:29:59.108 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package ceph-fuse. 2024-03-18T15:29:59.135 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-18T15:29:59.337 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking ceph-fuse (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:29:59.728 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package ceph-mds. 2024-03-18T15:29:59.754 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-18T15:29:59.794 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking ceph-mds (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:00.223 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package cephadm. 2024-03-18T15:30:00.250 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../071-cephadm_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-18T15:30:00.281 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking cephadm (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:00.567 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-asyncssh. 2024-03-18T15:30:00.594 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-03-18T15:30:00.633 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-03-18T15:30:00.938 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python-babel-localedata. 2024-03-18T15:30:00.952 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-03-18T15:30:00.985 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-18T15:30:01.073 INFO:teuthology.orchestra.run.smithi080.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-2162-g3d78f5ba-1jammy [14.2 kB] 2024-03-18T15:30:01.074 INFO:teuthology.orchestra.run.smithi080.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-2162-g3d78f5ba-1jammy [50.2 MB] 2024-03-18T15:30:01.968 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-babel. 2024-03-18T15:30:01.995 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-03-18T15:30:02.034 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-03-18T15:30:02.337 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-jinja2. 2024-03-18T15:30:02.364 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-03-18T15:30:02.403 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-18T15:30:02.740 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-03-18T15:30:02.767 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-2162-g3d78f5ba-1jammy_all.deb ... 2024-03-18T15:30:02.806 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking ceph-mgr-cephadm (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:03.101 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-repoze.lru. 2024-03-18T15:30:03.128 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-03-18T15:30:03.192 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-03-18T15:30:03.495 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-routes. 2024-03-18T15:30:03.522 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-03-18T15:30:03.562 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-03-18T15:30:03.907 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-03-18T15:30:03.934 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-2162-g3d78f5ba-1jammy_all.deb ... 2024-03-18T15:30:04.006 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking ceph-mgr-dashboard (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:04.618 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-03-18T15:30:04.646 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-03-18T15:30:04.684 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-18T15:30:05.138 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-joblib. 2024-03-18T15:30:05.154 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-03-18T15:30:05.188 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-03-18T15:30:05.558 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-03-18T15:30:05.586 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-03-18T15:30:05.710 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-03-18T15:30:05.971 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-sklearn. 2024-03-18T15:30:05.998 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-03-18T15:30:06.029 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-18T15:30:06.688 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-03-18T15:30:06.716 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-2162-g3d78f5ba-1jammy_all.deb ... 2024-03-18T15:30:06.771 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:07.708 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-cachetools. 2024-03-18T15:30:07.732 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-03-18T15:30:07.766 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-03-18T15:30:08.052 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-pyasn1. 2024-03-18T15:30:08.080 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-03-18T15:30:08.119 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-03-18T15:30:08.456 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-03-18T15:30:08.483 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-03-18T15:30:08.521 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-03-18T15:30:08.833 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-rsa. 2024-03-18T15:30:08.861 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-03-18T15:30:08.900 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-rsa (4.8-1) ... 2024-03-18T15:30:09.212 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-google-auth. 2024-03-18T15:30:09.239 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-03-18T15:30:09.278 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-03-18T15:30:09.582 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-03-18T15:30:09.609 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-03-18T15:30:09.648 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-18T15:30:09.934 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-websocket. 2024-03-18T15:30:09.950 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-03-18T15:30:09.984 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-03-18T15:30:10.262 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-kubernetes. 2024-03-18T15:30:10.290 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-03-18T15:30:10.339 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-18T15:30:10.951 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-03-18T15:30:10.968 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-2162-g3d78f5ba-1jammy_all.deb ... 2024-03-18T15:30:11.009 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:11.328 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package libonig5:amd64. 2024-03-18T15:30:11.356 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-03-18T15:30:11.395 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-18T15:30:11.748 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package libjq1:amd64. 2024-03-18T15:30:11.777 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-03-18T15:30:11.815 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-18T15:30:12.034 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package jq. 2024-03-18T15:30:12.062 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-03-18T15:30:12.092 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-03-18T15:30:12.345 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package socat. 2024-03-18T15:30:12.361 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-03-18T15:30:12.403 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-03-18T15:30:12.714 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package xmlstarlet. 2024-03-18T15:30:12.742 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-03-18T15:30:12.789 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-03-18T15:30:13.110 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package ceph-test. 2024-03-18T15:30:13.127 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../099-ceph-test_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-18T15:30:13.160 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking ceph-test (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:15.209 INFO:teuthology.orchestra.run.smithi080.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-2162-g3d78f5ba-1jammy [121 kB] 2024-03-18T15:30:15.212 INFO:teuthology.orchestra.run.smithi080.stdout:Get:119 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-2162-g3d78f5ba-1jammy [230 kB] 2024-03-18T15:30:15.271 INFO:teuthology.orchestra.run.smithi080.stdout:Get:120 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-2162-g3d78f5ba-1jammy [34.0 kB] 2024-03-18T15:30:15.327 INFO:teuthology.orchestra.run.smithi080.stdout:Get:121 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-2162-g3d78f5ba-1jammy [23.4 kB] 2024-03-18T15:30:15.329 INFO:teuthology.orchestra.run.smithi080.stdout:Get:122 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-2162-g3d78f5ba-1jammy [39.0 kB] 2024-03-18T15:30:15.330 INFO:teuthology.orchestra.run.smithi080.stdout:Get:123 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-2162-g3d78f5ba-1jammy [13.1 MB] 2024-03-18T15:30:15.548 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package ceph-volume. 2024-03-18T15:30:15.576 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-2162-g3d78f5ba-1jammy_all.deb ... 2024-03-18T15:30:15.614 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking ceph-volume (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:15.976 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package cephfs-mirror. 2024-03-18T15:30:16.004 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-18T15:30:16.043 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking cephfs-mirror (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:16.363 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-colorama. 2024-03-18T15:30:16.390 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-03-18T15:30:16.429 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-03-18T15:30:16.698 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package cephfs-shell. 2024-03-18T15:30:16.726 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-2162-g3d78f5ba-1jammy_all.deb ... 2024-03-18T15:30:16.765 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking cephfs-shell (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:17.034 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package cephfs-top. 2024-03-18T15:30:17.062 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-2162-g3d78f5ba-1jammy_all.deb ... 2024-03-18T15:30:17.100 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking cephfs-top (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:17.352 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package libcephfs-dev. 2024-03-18T15:30:17.370 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-18T15:30:17.411 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking libcephfs-dev (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:17.705 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package lua-socket:amd64. 2024-03-18T15:30:17.722 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-03-18T15:30:17.755 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-18T15:30:18.084 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package lua-sec:amd64. 2024-03-18T15:30:18.103 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-03-18T15:30:18.143 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-03-18T15:30:18.522 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package nvme-cli. 2024-03-18T15:30:18.549 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-03-18T15:30:18.588 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-03-18T15:30:18.985 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package pkg-config. 2024-03-18T15:30:19.013 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-03-18T15:30:19.051 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-03-18T15:30:19.287 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-03-18T15:30:19.304 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-03-18T15:30:19.337 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-03-18T15:30:19.749 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-iniconfig. 2024-03-18T15:30:19.777 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-03-18T15:30:19.890 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-03-18T15:30:20.218 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-pastescript. 2024-03-18T15:30:20.246 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-03-18T15:30:20.284 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-03-18T15:30:20.596 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-pluggy. 2024-03-18T15:30:20.624 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-03-18T15:30:20.662 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-03-18T15:30:20.949 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-psutil. 2024-03-18T15:30:20.977 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-03-18T15:30:21.007 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-03-18T15:30:21.352 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-py. 2024-03-18T15:30:21.380 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-03-18T15:30:21.418 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-py (1.10.0-1) ... 2024-03-18T15:30:21.575 INFO:teuthology.orchestra.run.smithi080.stdout:Get:124 https://2.chacra.ceph.com/r/ceph/wip-batrick-testing-20240314.183243/3d78f5ba07bf18880172c07d4d7b4ac00c9d30af/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-2162-g3d78f5ba-1jammy [91.5 kB] 2024-03-18T15:30:21.721 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-pygments. 2024-03-18T15:30:21.738 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-03-18T15:30:21.779 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-03-18T15:30:22.178 INFO:teuthology.orchestra.run.smithi080.stdout:Fetched 178 MB in 1min 1s (2,905 kB/s) 2024-03-18T15:30:22.209 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-pyinotify. 2024-03-18T15:30:22.237 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-03-18T15:30:22.266 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-03-18T15:30:22.320 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-pyperclip. 2024-03-18T15:30:22.417 INFO:teuthology.orchestra.run.smithi080.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165997 files and directories currently installed.) 2024-03-18T15:30:22.422 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-03-18T15:30:22.468 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-03-18T15:30:22.569 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-toml. 2024-03-18T15:30:22.597 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-03-18T15:30:22.636 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-03-18T15:30:22.738 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-wcwidth. 2024-03-18T15:30:22.764 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-03-18T15:30:22.813 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-18T15:30:22.938 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-pytest. 2024-03-18T15:30:22.967 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-03-18T15:30:23.005 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-03-18T15:30:23.107 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-cmd2. 2024-03-18T15:30:23.123 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-03-18T15:30:23.166 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-03-18T15:30:23.326 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package python3-simplejson. 2024-03-18T15:30:23.354 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-03-18T15:30:23.391 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-03-18T15:30:23.486 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-03-18T15:30:23.512 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-03-18T15:30:23.594 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-18T15:30:23.653 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-03-18T15:30:23.681 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-03-18T15:30:23.719 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-03-18T15:30:23.948 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-03-18T15:30:23.974 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-03-18T15:30:24.014 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-18T15:30:24.317 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-03-18T15:30:24.343 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-03-18T15:30:24.376 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-18T15:30:24.498 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package radosgw. 2024-03-18T15:30:24.526 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../122-radosgw_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-18T15:30:24.564 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking radosgw (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:24.813 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-03-18T15:30:24.839 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-18T15:30:24.887 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-18T15:30:25.291 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-03-18T15:30:25.317 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-18T15:30:25.357 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-18T15:30:25.367 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package rbd-fuse. 2024-03-18T15:30:25.384 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-18T15:30:25.417 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking rbd-fuse (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:25.669 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-03-18T15:30:25.695 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-18T15:30:25.714 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-03-18T15:30:25.735 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-18T15:30:25.972 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-18T15:30:26.090 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-03-18T15:30:26.114 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-03-18T15:30:26.140 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-03-18T15:30:26.180 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-18T15:30:26.356 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-03-18T15:30:26.534 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package librados2. 2024-03-18T15:30:26.560 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../010-librados2_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-18T15:30:26.600 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking librados2 (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:26.701 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-03-18T15:30:27.119 INFO:teuthology.orchestra.run.smithi005.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-03-18T15:30:27.184 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up cephadm (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:27.193 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package librbd1. 2024-03-18T15:30:27.220 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../011-librbd1_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-18T15:30:27.253 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking librbd1 (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:27.697 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libcephfs2. 2024-03-18T15:30:27.724 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-18T15:30:27.764 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libcephfs2 (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:27.833 INFO:teuthology.orchestra.run.smithi005.stdout:Adding system user cephadm....done 2024-03-18T15:30:27.981 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-18T15:30:28.042 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-rados. 2024-03-18T15:30:28.057 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../013-python3-rados_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-18T15:30:28.100 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-rados (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:28.291 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-03-18T15:30:28.394 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-03-18T15:30:28.421 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-2162-g3d78f5ba-1jammy_all.deb ... 2024-03-18T15:30:28.461 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-ceph-argparse (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:28.534 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-03-18T15:30:28.660 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-03-18T15:30:28.689 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-cephfs. 2024-03-18T15:30:28.715 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-18T15:30:28.755 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-cephfs (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:29.002 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-03-18T15:30:29.092 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-ceph-common. 2024-03-18T15:30:29.118 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-2162-g3d78f5ba-1jammy_all.deb ... 2024-03-18T15:30:29.158 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-ceph-common (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:29.270 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-03-18T15:30:29.371 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-py (1.10.0-1) ... 2024-03-18T15:30:29.486 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-prettytable. 2024-03-18T15:30:29.513 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-03-18T15:30:29.553 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-03-18T15:30:29.690 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-03-18T15:30:29.789 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-rbd. 2024-03-18T15:30:29.815 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-18T15:30:29.855 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-rbd (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:30.050 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-03-18T15:30:30.167 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-03-18T15:30:30.193 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-03-18T15:30:30.233 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-18T15:30:30.293 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-03-18T15:30:30.511 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-03-18T15:30:30.537 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-03-18T15:30:30.561 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-03-18T15:30:30.578 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-03-18T15:30:30.837 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-03-18T15:30:30.880 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-03-18T15:30:30.888 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-03-18T15:30:30.931 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-18T15:30:31.047 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-03-18T15:30:31.252 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package lua5.1. 2024-03-18T15:30:31.277 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-03-18T15:30:31.290 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-03-18T15:30:31.309 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-03-18T15:30:31.549 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-03-18T15:30:31.620 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package lua-any. 2024-03-18T15:30:31.636 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-03-18T15:30:31.680 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking lua-any (27ubuntu1) ... 2024-03-18T15:30:31.931 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package zip. 2024-03-18T15:30:31.958 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-03-18T15:30:31.959 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-ceph-argparse (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:31.998 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking zip (3.0-12build2) ... 2024-03-18T15:30:32.218 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-18T15:30:32.327 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package unzip. 2024-03-18T15:30:32.344 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-18T15:30:32.354 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-03-18T15:30:32.394 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-03-18T15:30:32.470 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-03-18T15:30:32.737 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-03-18T15:30:32.747 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package luarocks. 2024-03-18T15:30:32.773 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-03-18T15:30:32.813 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-03-18T15:30:32.863 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-03-18T15:30:33.131 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-03-18T15:30:33.188 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package librgw2. 2024-03-18T15:30:33.214 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../027-librgw2_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-18T15:30:33.255 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking librgw2 (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:33.501 INFO:teuthology.orchestra.run.smithi005.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-03-18T15:30:33.529 INFO:teuthology.orchestra.run.smithi005.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-03-18T15:30:33.593 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-18T15:30:33.719 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-03-18T15:30:33.832 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-rgw. 2024-03-18T15:30:33.859 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-18T15:30:33.899 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-rgw (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:34.086 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-18T15:30:34.201 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package liboath0:amd64. 2024-03-18T15:30:34.212 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-03-18T15:30:34.230 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-03-18T15:30:34.260 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-03-18T15:30:34.480 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-03-18T15:30:34.571 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libradosstriper1. 2024-03-18T15:30:34.597 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-18T15:30:34.638 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libradosstriper1 (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:34.731 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-03-18T15:30:35.024 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up zip (3.0-12build2) ... 2024-03-18T15:30:35.050 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-common. 2024-03-18T15:30:35.076 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../031-ceph-common_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-18T15:30:35.116 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-common (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:35.125 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-03-18T15:30:35.777 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-03-18T15:30:36.036 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-03-18T15:30:36.162 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-03-18T15:30:36.290 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-18T15:30:36.446 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-base. 2024-03-18T15:30:36.473 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../032-ceph-base_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-18T15:30:36.523 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-base (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:36.597 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-03-18T15:30:36.999 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-03-18T15:30:37.059 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-03-18T15:30:37.086 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-03-18T15:30:37.126 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-03-18T15:30:37.233 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-03-18T15:30:37.345 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-cheroot. 2024-03-18T15:30:37.361 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-03-18T15:30:37.395 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-18T15:30:37.618 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-18T15:30:37.690 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-03-18T15:30:37.717 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-03-18T15:30:37.756 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-03-18T15:30:37.911 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-03-18T15:30:38.068 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-jaraco.text. 2024-03-18T15:30:38.095 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-03-18T15:30:38.134 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-03-18T15:30:38.220 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-18T15:30:38.437 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-03-18T15:30:38.464 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-03-18T15:30:38.496 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-03-18T15:30:38.563 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-03-18T15:30:38.757 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-tempora. 2024-03-18T15:30:38.783 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-03-18T15:30:38.806 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-03-18T15:30:38.832 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-03-18T15:30:38.915 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-ceph-common (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:39.135 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-portend. 2024-03-18T15:30:39.162 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-03-18T15:30:39.191 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-idna (3.3-1) ... 2024-03-18T15:30:39.201 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-03-18T15:30:39.479 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-03-18T15:30:39.501 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-18T15:30:39.506 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-03-18T15:30:39.546 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-03-18T15:30:39.791 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-cherrypy3. 2024-03-18T15:30:39.817 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-03-18T15:30:39.849 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-03-18T15:30:40.194 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-natsort. 2024-03-18T15:30:40.221 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-03-18T15:30:40.261 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-03-18T15:30:40.631 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-logutils. 2024-03-18T15:30:40.657 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-03-18T15:30:40.697 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-03-18T15:30:40.717 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-03-18T15:30:40.942 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-18T15:30:40.967 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-markupsafe. 2024-03-18T15:30:40.994 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-03-18T15:30:41.025 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-03-18T15:30:41.093 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-toml (0.10.2-1) ... 2024-03-18T15:30:41.320 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-mako. 2024-03-18T15:30:41.346 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-03-18T15:30:41.360 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-18T15:30:41.403 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-18T15:30:41.486 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-03-18T15:30:41.612 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-18T15:30:41.715 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-simplegeneric. 2024-03-18T15:30:41.741 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-03-18T15:30:41.781 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-03-18T15:30:41.897 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-03-18T15:30:42.068 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-singledispatch. 2024-03-18T15:30:42.094 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-03-18T15:30:42.134 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-03-18T15:30:42.165 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-03-18T15:30:42.399 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-18T15:30:42.437 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-webob. 2024-03-18T15:30:42.463 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-03-18T15:30:42.500 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-03-18T15:30:42.503 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-03-18T15:30:42.793 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-03-18T15:30:42.848 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-waitress. 2024-03-18T15:30:42.874 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-03-18T15:30:42.918 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-18T15:30:43.044 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-03-18T15:30:43.226 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-tempita. 2024-03-18T15:30:43.253 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-03-18T15:30:43.293 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-03-18T15:30:43.295 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-03-18T15:30:43.521 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-03-18T15:30:43.546 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-paste. 2024-03-18T15:30:43.561 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-03-18T15:30:43.596 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-03-18T15:30:43.764 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-03-18T15:30:43.924 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-03-18T15:30:43.950 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-03-18T15:30:43.991 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-03-18T15:30:44.032 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-18T15:30:44.133 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-03-18T15:30:44.293 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-pastedeploy. 2024-03-18T15:30:44.320 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-03-18T15:30:44.360 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-03-18T15:30:44.401 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-18T15:30:44.527 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-18T15:30:44.680 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-webtest. 2024-03-18T15:30:44.706 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-03-18T15:30:44.746 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-03-18T15:30:44.836 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-03-18T15:30:45.016 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-pecan. 2024-03-18T15:30:45.043 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-03-18T15:30:45.074 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-03-18T15:30:45.154 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-03-18T15:30:45.396 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-certifi. 2024-03-18T15:30:45.396 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-18T15:30:45.412 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-03-18T15:30:45.454 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-03-18T15:30:45.522 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-03-18T15:30:45.669 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-03-18T15:30:45.782 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-idna. 2024-03-18T15:30:45.809 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-03-18T15:30:45.849 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-idna (3.3-1) ... 2024-03-18T15:30:46.156 INFO:teuthology.orchestra.run.smithi005.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-03-18T15:30:46.160 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-urllib3. 2024-03-18T15:30:46.187 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-03-18T15:30:46.227 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-18T15:30:46.427 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-18T15:30:46.553 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-03-18T15:30:46.555 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-requests. 2024-03-18T15:30:46.582 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-03-18T15:30:46.621 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-18T15:30:46.933 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-werkzeug. 2024-03-18T15:30:46.946 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-03-18T15:30:46.960 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-03-18T15:30:47.000 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-18T15:30:47.197 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up lua-any (27ubuntu1) ... 2024-03-18T15:30:47.286 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-03-18T15:30:47.313 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-2162-g3d78f5ba-1jammy_all.deb ... 2024-03-18T15:30:47.323 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-portend (3.0.0-1) ... 2024-03-18T15:30:47.352 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-mgr-modules-core (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:47.574 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-18T15:30:47.684 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-18T15:30:47.783 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-03-18T15:30:47.810 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-18T15:30:47.849 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:48.010 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-03-18T15:30:48.160 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-bcrypt. 2024-03-18T15:30:48.187 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-03-18T15:30:48.227 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-03-18T15:30:48.320 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-18T15:30:48.505 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-openssl. 2024-03-18T15:30:48.532 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-03-18T15:30:48.563 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-03-18T15:30:48.587 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-03-18T15:30:48.798 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-rsa (4.8-1) ... 2024-03-18T15:30:48.900 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-mgr. 2024-03-18T15:30:48.926 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-18T15:30:48.966 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-mgr (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:49.050 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-03-18T15:30:49.318 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-18T15:30:49.361 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-mon. 2024-03-18T15:30:49.388 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-18T15:30:49.427 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-mon (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:49.578 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-03-18T15:30:49.954 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-03-18T15:30:50.032 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-osd. 2024-03-18T15:30:50.058 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-18T15:30:50.097 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-osd (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:50.222 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-03-18T15:30:50.575 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-18T15:30:50.709 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up librados2 (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:50.860 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up libsqlite3-mod-ceph (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:51.011 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-03-18T15:30:51.101 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph. 2024-03-18T15:30:51.117 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../068-ceph_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-18T15:30:51.151 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:51.348 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up libcephfs2 (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:51.465 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up libradosstriper1 (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:51.487 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-fuse. 2024-03-18T15:30:51.513 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-18T15:30:51.553 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-fuse (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:51.567 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-03-18T15:30:51.835 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up librbd1 (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:51.890 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-mds. 2024-03-18T15:30:51.917 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-18T15:30:51.948 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-mds (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:51.986 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up ceph-mgr-modules-core (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:52.137 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up ceph-fuse (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:52.351 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package cephadm. 2024-03-18T15:30:52.357 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-18T15:30:52.357 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-18T15:30:52.378 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../071-cephadm_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-18T15:30:52.417 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking cephadm (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:52.729 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-asyncssh. 2024-03-18T15:30:52.756 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-03-18T15:30:52.795 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-03-18T15:30:52.855 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up libcephfs-dev (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:52.981 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-rados (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:53.106 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python-babel-localedata. 2024-03-18T15:30:53.107 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up librgw2 (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:53.133 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-03-18T15:30:53.173 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-18T15:30:53.208 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up cephfs-top (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:53.317 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-18T15:30:54.144 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-babel. 2024-03-18T15:30:54.160 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-03-18T15:30:54.194 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-03-18T15:30:54.472 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-jinja2. 2024-03-18T15:30:54.488 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-03-18T15:30:54.522 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-18T15:30:54.629 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-rbd (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:54.755 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up rbd-fuse (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:54.809 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-03-18T15:30:54.836 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-2162-g3d78f5ba-1jammy_all.deb ... 2024-03-18T15:30:54.875 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-mgr-cephadm (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:54.881 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-rgw (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:54.999 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up python3-cephfs (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:55.100 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up cephfs-shell (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:55.162 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-repoze.lru. 2024-03-18T15:30:55.177 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-03-18T15:30:55.212 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-03-18T15:30:55.226 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up ceph-common (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:55.515 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-routes. 2024-03-18T15:30:55.542 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-03-18T15:30:55.581 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-03-18T15:30:55.786 INFO:teuthology.orchestra.run.smithi005.stdout:Adding group ceph....done 2024-03-18T15:30:55.935 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-03-18T15:30:55.962 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-2162-g3d78f5ba-1jammy_all.deb ... 2024-03-18T15:30:56.001 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-mgr-dashboard (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:56.170 INFO:teuthology.orchestra.run.smithi005.stdout:Adding system user ceph....done 2024-03-18T15:30:56.331 INFO:teuthology.orchestra.run.smithi005.stdout:Setting system user ceph properties....done 2024-03-18T15:30:56.349 INFO:teuthology.orchestra.run.smithi005.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-03-18T15:30:56.438 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-03-18T15:30:56.465 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-03-18T15:30:56.499 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-03-18T15:30:56.504 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-18T15:30:56.824 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-03-18T15:30:57.025 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-joblib. 2024-03-18T15:30:57.042 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-03-18T15:30:57.083 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-03-18T15:30:57.286 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up cephfs-mirror (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:57.412 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-03-18T15:30:57.439 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-03-18T15:30:57.502 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-03-18T15:30:57.503 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-03-18T15:30:57.564 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-03-18T15:30:57.875 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-sklearn. 2024-03-18T15:30:57.902 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-03-18T15:30:57.942 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-18T15:30:57.990 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up ceph-test (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:58.113 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up radosgw (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:58.586 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-18T15:30:58.586 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-18T15:30:58.945 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-03-18T15:30:58.972 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-2162-g3d78f5ba-1jammy_all.deb ... 2024-03-18T15:30:59.011 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:59.048 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up ceph-base (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:59.390 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-03-18T15:30:59.883 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up ceph-mds (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:30:59.997 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-cachetools. 2024-03-18T15:31:00.025 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-03-18T15:31:00.056 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-03-18T15:31:00.094 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-18T15:31:00.094 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-18T15:31:00.334 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-pyasn1. 2024-03-18T15:31:00.361 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-03-18T15:31:00.409 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-03-18T15:31:00.561 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up ceph-mgr (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:00.746 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-03-18T15:31:00.772 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-03-18T15:31:00.789 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-18T15:31:00.789 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-18T15:31:00.812 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-03-18T15:31:01.090 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-rsa. 2024-03-18T15:31:01.118 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-03-18T15:31:01.157 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-rsa (4.8-1) ... 2024-03-18T15:31:01.270 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up ceph-osd (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:01.444 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-google-auth. 2024-03-18T15:31:01.472 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-03-18T15:31:01.511 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-03-18T15:31:01.561 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-18T15:31:01.561 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-18T15:31:01.831 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-03-18T15:31:01.858 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-03-18T15:31:01.897 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-18T15:31:02.043 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up ceph-mgr-k8sevents (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:02.169 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:02.209 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-websocket. 2024-03-18T15:31:02.236 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-03-18T15:31:02.275 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-03-18T15:31:02.320 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up ceph-mon (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:02.539 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-18T15:31:02.539 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-18T15:31:02.579 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-kubernetes. 2024-03-18T15:31:02.606 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-03-18T15:31:02.637 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-18T15:31:03.071 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up ceph-mgr-cephadm (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:03.222 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up ceph (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:03.292 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-03-18T15:31:03.308 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-2162-g3d78f5ba-1jammy_all.deb ... 2024-03-18T15:31:03.348 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up ceph-mgr-dashboard (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:03.359 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:03.490 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up ceph-volume (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:03.687 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libonig5:amd64. 2024-03-18T15:31:03.714 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-03-18T15:31:03.753 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-18T15:31:03.852 INFO:teuthology.orchestra.run.smithi005.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-03-18T15:31:03.961 INFO:teuthology.orchestra.run.smithi005.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-18T15:31:04.065 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libjq1:amd64. 2024-03-18T15:31:04.092 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-03-18T15:31:04.123 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-18T15:31:04.189 INFO:teuthology.orchestra.run.smithi005.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-18T15:31:04.376 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package jq. 2024-03-18T15:31:04.391 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-03-18T15:31:04.442 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-03-18T15:31:04.695 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package socat. 2024-03-18T15:31:04.722 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-03-18T15:31:04.762 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-03-18T15:31:05.065 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package xmlstarlet. 2024-03-18T15:31:05.081 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-03-18T15:31:05.123 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-03-18T15:31:05.331 INFO:teuthology.orchestra.run.smithi005.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-18T15:31:05.486 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-test. 2024-03-18T15:31:05.514 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../099-ceph-test_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-18T15:31:05.553 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-test (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:05.983 INFO:teuthology.orchestra.run.smithi005.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-18T15:31:05.986 DEBUG:teuthology.orchestra.run.smithi005:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-03-18T15:31:06.050 INFO:teuthology.orchestra.run.smithi005.stdout:Reading package lists... 2024-03-18T15:31:06.251 INFO:teuthology.orchestra.run.smithi005.stdout:Building dependency tree... 2024-03-18T15:31:06.251 INFO:teuthology.orchestra.run.smithi005.stdout:Reading state information... 2024-03-18T15:31:06.436 INFO:teuthology.orchestra.run.smithi005.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-03-18T15:31:06.436 INFO:teuthology.orchestra.run.smithi005.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-03-18T15:31:06.436 INFO:teuthology.orchestra.run.smithi005.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2024-03-18T15:31:06.437 INFO:teuthology.orchestra.run.smithi005.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2024-03-18T15:31:06.437 INFO:teuthology.orchestra.run.smithi005.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-03-18T15:31:06.437 INFO:teuthology.orchestra.run.smithi005.stdout:The following packages were automatically installed and are no longer required: 2024-03-18T15:31:06.437 INFO:teuthology.orchestra.run.smithi005.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-18T15:31:06.437 INFO:teuthology.orchestra.run.smithi005.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-18T15:31:06.437 INFO:teuthology.orchestra.run.smithi005.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-18T15:31:06.437 INFO:teuthology.orchestra.run.smithi005.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-18T15:31:06.438 INFO:teuthology.orchestra.run.smithi005.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-18T15:31:06.438 INFO:teuthology.orchestra.run.smithi005.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-18T15:31:06.440 INFO:teuthology.orchestra.run.smithi005.stdout:The following additional packages will be installed: 2024-03-18T15:31:06.440 INFO:teuthology.orchestra.run.smithi005.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-03-18T15:31:06.440 INFO:teuthology.orchestra.run.smithi005.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-03-18T15:31:06.440 INFO:teuthology.orchestra.run.smithi005.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2024-03-18T15:31:06.440 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-03-18T15:31:06.440 INFO:teuthology.orchestra.run.smithi005.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-03-18T15:31:06.441 INFO:teuthology.orchestra.run.smithi005.stdout:Suggested packages: 2024-03-18T15:31:06.441 INFO:teuthology.orchestra.run.smithi005.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-03-18T15:31:06.441 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2024-03-18T15:31:06.441 INFO:teuthology.orchestra.run.smithi005.stdout: libengine-pkcs11-openssl 2024-03-18T15:31:06.480 INFO:teuthology.orchestra.run.smithi005.stdout:The following NEW packages will be installed: 2024-03-18T15:31:06.481 INFO:teuthology.orchestra.run.smithi005.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-03-18T15:31:06.481 INFO:teuthology.orchestra.run.smithi005.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-03-18T15:31:06.481 INFO:teuthology.orchestra.run.smithi005.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-03-18T15:31:06.482 INFO:teuthology.orchestra.run.smithi005.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-03-18T15:31:06.482 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-03-18T15:31:06.482 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-03-18T15:31:06.482 INFO:teuthology.orchestra.run.smithi005.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-03-18T15:31:06.483 INFO:teuthology.orchestra.run.smithi005.stdout:The following packages will be upgraded: 2024-03-18T15:31:06.484 INFO:teuthology.orchestra.run.smithi005.stdout: libssl3 2024-03-18T15:31:06.575 INFO:teuthology.orchestra.run.smithi005.stdout:1 upgraded, 39 newly installed, 0 to remove and 284 not upgraded. 2024-03-18T15:31:06.575 INFO:teuthology.orchestra.run.smithi005.stdout:Need to get 54.6 MB of archives. 2024-03-18T15:31:06.575 INFO:teuthology.orchestra.run.smithi005.stdout:After this operation, 202 MB of additional disk space will be used. 2024-03-18T15:31:06.575 INFO:teuthology.orchestra.run.smithi005.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-03-18T15:31:06.703 INFO:teuthology.orchestra.run.smithi005.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-03-18T15:31:06.774 INFO:teuthology.orchestra.run.smithi005.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-03-18T15:31:06.791 INFO:teuthology.orchestra.run.smithi005.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-03-18T15:31:06.792 INFO:teuthology.orchestra.run.smithi005.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-03-18T15:31:06.795 INFO:teuthology.orchestra.run.smithi005.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-03-18T15:31:06.797 INFO:teuthology.orchestra.run.smithi005.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-03-18T15:31:06.798 INFO:teuthology.orchestra.run.smithi005.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-03-18T15:31:06.800 INFO:teuthology.orchestra.run.smithi005.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-03-18T15:31:06.801 INFO:teuthology.orchestra.run.smithi005.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-03-18T15:31:06.802 INFO:teuthology.orchestra.run.smithi005.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-03-18T15:31:06.802 INFO:teuthology.orchestra.run.smithi005.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-03-18T15:31:06.821 INFO:teuthology.orchestra.run.smithi005.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-03-18T15:31:07.237 INFO:teuthology.orchestra.run.smithi005.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-03-18T15:31:07.239 INFO:teuthology.orchestra.run.smithi005.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-03-18T15:31:07.239 INFO:teuthology.orchestra.run.smithi005.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-03-18T15:31:07.243 INFO:teuthology.orchestra.run.smithi005.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-03-18T15:31:07.243 INFO:teuthology.orchestra.run.smithi005.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-03-18T15:31:07.249 INFO:teuthology.orchestra.run.smithi005.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-03-18T15:31:07.251 INFO:teuthology.orchestra.run.smithi005.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-03-18T15:31:07.261 INFO:teuthology.orchestra.run.smithi005.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-03-18T15:31:07.261 INFO:teuthology.orchestra.run.smithi005.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-03-18T15:31:07.289 INFO:teuthology.orchestra.run.smithi005.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-03-18T15:31:07.289 INFO:teuthology.orchestra.run.smithi005.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-03-18T15:31:07.301 INFO:teuthology.orchestra.run.smithi005.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-03-18T15:31:07.325 INFO:teuthology.orchestra.run.smithi005.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-03-18T15:31:07.332 INFO:teuthology.orchestra.run.smithi005.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-03-18T15:31:07.333 INFO:teuthology.orchestra.run.smithi005.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-03-18T15:31:07.333 INFO:teuthology.orchestra.run.smithi005.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-03-18T15:31:07.334 INFO:teuthology.orchestra.run.smithi005.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-03-18T15:31:07.360 INFO:teuthology.orchestra.run.smithi005.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-03-18T15:31:07.362 INFO:teuthology.orchestra.run.smithi005.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-03-18T15:31:07.706 INFO:teuthology.orchestra.run.smithi005.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-03-18T15:31:07.719 INFO:teuthology.orchestra.run.smithi005.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-03-18T15:31:07.720 INFO:teuthology.orchestra.run.smithi005.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-03-18T15:31:07.722 INFO:teuthology.orchestra.run.smithi005.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-03-18T15:31:07.723 INFO:teuthology.orchestra.run.smithi005.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-03-18T15:31:07.725 INFO:teuthology.orchestra.run.smithi005.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-03-18T15:31:07.748 INFO:teuthology.orchestra.run.smithi005.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-03-18T15:31:07.768 INFO:teuthology.orchestra.run.smithi005.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-03-18T15:31:07.798 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-volume. 2024-03-18T15:31:07.814 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-2162-g3d78f5ba-1jammy_all.deb ... 2024-03-18T15:31:07.848 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-volume (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:08.178 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package cephfs-mirror. 2024-03-18T15:31:08.206 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-18T15:31:08.220 INFO:teuthology.orchestra.run.smithi005.stdout:Preconfiguring packages ... 2024-03-18T15:31:08.235 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking cephfs-mirror (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:08.384 INFO:teuthology.orchestra.run.smithi005.stdout:Fetched 54.6 MB in 1s (41.9 MB/s) 2024-03-18T15:31:08.514 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-colorama. 2024-03-18T15:31:08.529 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package flex. 2024-03-18T15:31:08.541 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-03-18T15:31:08.580 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-03-18T15:31:08.635 INFO:teuthology.orchestra.run.smithi005.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173660 files and directories currently installed.) 2024-03-18T15:31:08.640 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-03-18T15:31:08.686 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking flex (2.6.4-8build2) ... 2024-03-18T15:31:08.841 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package cephfs-shell. 2024-03-18T15:31:08.857 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-2162-g3d78f5ba-1jammy_all.deb ... 2024-03-18T15:31:08.899 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking cephfs-shell (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:09.202 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package cephfs-top. 2024-03-18T15:31:09.229 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-2162-g3d78f5ba-1jammy_all.deb ... 2024-03-18T15:31:09.253 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-03-18T15:31:09.269 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking cephfs-top (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:09.417 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-03-18T15:31:09.513 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libcephfs-dev. 2024-03-18T15:31:09.528 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-18T15:31:09.562 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libcephfs-dev (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:09.815 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package lua-socket:amd64. 2024-03-18T15:31:09.828 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-18T15:31:09.832 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-03-18T15:31:09.865 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-18T15:31:10.170 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package lua-sec:amd64. 2024-03-18T15:31:10.198 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-03-18T15:31:10.228 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-03-18T15:31:10.367 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package bison. 2024-03-18T15:31:10.441 INFO:teuthology.orchestra.run.smithi005.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173733 files and directories currently installed.) 2024-03-18T15:31:10.446 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-03-18T15:31:10.495 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-03-18T15:31:10.591 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package nvme-cli. 2024-03-18T15:31:10.618 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-03-18T15:31:10.657 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-03-18T15:31:10.896 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-03-18T15:31:10.925 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-03-18T15:31:10.962 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-03-18T15:31:11.071 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package pkg-config. 2024-03-18T15:31:11.098 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-03-18T15:31:11.137 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-03-18T15:31:11.249 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package dump. 2024-03-18T15:31:11.277 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-03-18T15:31:11.307 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking dump (0.4b47-3) ... 2024-03-18T15:31:11.398 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-03-18T15:31:11.426 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-03-18T15:31:11.465 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-03-18T15:31:11.627 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package indent. 2024-03-18T15:31:11.655 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-03-18T15:31:11.694 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-03-18T15:31:11.869 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-iniconfig. 2024-03-18T15:31:11.896 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-03-18T15:31:11.935 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-03-18T15:31:11.963 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-03-18T15:31:11.991 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-03-18T15:31:12.029 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-03-18T15:31:12.263 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-pastescript. 2024-03-18T15:31:12.291 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-03-18T15:31:12.329 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-03-18T15:31:12.333 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-03-18T15:31:12.361 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-03-18T15:31:12.399 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-03-18T15:31:12.644 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-03-18T15:31:12.658 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-pluggy. 2024-03-18T15:31:12.660 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-03-18T15:31:12.686 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-03-18T15:31:12.702 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-03-18T15:31:12.724 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-03-18T15:31:13.003 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-psutil. 2024-03-18T15:31:13.030 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-03-18T15:31:13.047 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package libfl2:amd64. 2024-03-18T15:31:13.060 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-03-18T15:31:13.075 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-03-18T15:31:13.113 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-03-18T15:31:13.324 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-03-18T15:31:13.341 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-03-18T15:31:13.374 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-03-18T15:31:13.610 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package libjson-perl. 2024-03-18T15:31:13.611 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-py. 2024-03-18T15:31:13.638 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-03-18T15:31:13.639 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-03-18T15:31:13.677 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-03-18T15:31:13.697 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-py (1.10.0-1) ... 2024-03-18T15:31:13.980 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package libllvm14:amd64. 2024-03-18T15:31:14.008 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-03-18T15:31:14.025 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-pygments. 2024-03-18T15:31:14.046 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-03-18T15:31:14.052 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-03-18T15:31:14.091 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-03-18T15:31:14.529 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-pyinotify. 2024-03-18T15:31:14.557 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-03-18T15:31:14.596 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-03-18T15:31:14.898 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-toml. 2024-03-18T15:31:14.926 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-03-18T15:31:14.965 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-03-18T15:31:15.301 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-pytest. 2024-03-18T15:31:15.329 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-03-18T15:31:15.368 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-03-18T15:31:15.713 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-simplejson. 2024-03-18T15:31:15.740 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-03-18T15:31:15.779 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-03-18T15:31:15.807 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-03-18T15:31:15.836 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-03-18T15:31:15.874 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-03-18T15:31:16.049 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-03-18T15:31:16.065 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-03-18T15:31:16.107 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-03-18T15:31:16.110 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package libmbim-proxy. 2024-03-18T15:31:16.126 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-03-18T15:31:16.168 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-03-18T15:31:16.496 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-03-18T15:31:16.524 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-03-18T15:31:16.562 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-03-18T15:31:16.761 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package radosgw. 2024-03-18T15:31:16.789 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../122-radosgw_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-18T15:31:16.819 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking radosgw (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:16.932 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package libndp0:amd64. 2024-03-18T15:31:16.960 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-03-18T15:31:16.998 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-03-18T15:31:17.277 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package libnm0:amd64. 2024-03-18T15:31:17.304 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-03-18T15:31:17.334 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-03-18T15:31:17.642 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package rbd-fuse. 2024-03-18T15:31:17.667 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-2162-g3d78f5ba-1jammy_amd64.deb ... 2024-03-18T15:31:17.671 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package libpq5:amd64. 2024-03-18T15:31:17.700 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-18T15:31:17.705 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking rbd-fuse (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:17.737 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-03-18T15:31:18.008 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-03-18T15:31:18.082 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-03-18T15:31:18.110 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-03-18T15:31:18.148 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-03-18T15:31:18.261 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-18T15:31:18.387 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-03-18T15:31:18.442 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package libqmi-proxy. 2024-03-18T15:31:18.459 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-03-18T15:31:18.502 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-03-18T15:31:18.649 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-03-18T15:31:18.711 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-03-18T15:31:18.728 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-03-18T15:31:18.761 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-18T15:31:18.974 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-03-18T15:31:19.231 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-03-18T15:31:19.260 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-03-18T15:31:19.289 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-03-18T15:31:19.411 INFO:teuthology.orchestra.run.smithi080.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-03-18T15:31:19.609 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package modemmanager. 2024-03-18T15:31:19.638 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-03-18T15:31:19.675 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-03-18T15:31:19.707 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up cephadm (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:20.207 INFO:teuthology.orchestra.run.smithi080.stdout:Adding system user cephadm....done 2024-03-18T15:31:20.248 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package network-manager. 2024-03-18T15:31:20.277 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-03-18T15:31:20.315 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-03-18T15:31:20.363 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-18T15:31:20.639 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-03-18T15:31:20.865 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-03-18T15:31:20.956 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package ppp. 2024-03-18T15:31:20.983 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-03-18T15:31:20.984 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-03-18T15:31:21.234 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-03-18T15:31:21.254 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-03-18T15:31:21.511 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-03-18T15:31:21.569 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package pptp-linux. 2024-03-18T15:31:21.597 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-03-18T15:31:21.637 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-py (1.10.0-1) ... 2024-03-18T15:31:21.643 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-03-18T15:31:21.938 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package network-manager-pptp. 2024-03-18T15:31:21.947 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-03-18T15:31:21.955 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-03-18T15:31:21.996 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-03-18T15:31:22.290 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-03-18T15:31:22.299 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package postgresql-client-common. 2024-03-18T15:31:22.327 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-03-18T15:31:22.366 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking postgresql-client-common (238) ... 2024-03-18T15:31:22.542 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-03-18T15:31:22.703 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package postgresql-client-14. 2024-03-18T15:31:22.731 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-18T15:31:22.794 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-03-18T15:31:22.803 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-18T15:31:23.069 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-03-18T15:31:23.312 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-03-18T15:31:23.434 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package postgresql-common. 2024-03-18T15:31:23.461 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-03-18T15:31:23.519 INFO:teuthology.orchestra.run.smithi005.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-03-18T15:31:23.548 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking postgresql-common (238) ... 2024-03-18T15:31:23.555 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-03-18T15:31:23.789 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-03-18T15:31:23.998 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package postgresql-14. 2024-03-18T15:31:24.027 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-18T15:31:24.068 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-18T15:31:24.073 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-ceph-argparse (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:24.342 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-18T15:31:24.484 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-18T15:31:24.614 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-03-18T15:31:24.870 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-03-18T15:31:24.971 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-03-18T15:31:25.205 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-03-18T15:31:25.476 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package postgresql. 2024-03-18T15:31:25.494 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-03-18T15:31:25.534 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking postgresql (14+238) ... 2024-03-18T15:31:25.592 INFO:teuthology.orchestra.run.smithi080.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-03-18T15:31:25.620 INFO:teuthology.orchestra.run.smithi080.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-03-18T15:31:25.684 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-18T15:31:25.770 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package postgresql-client. 2024-03-18T15:31:25.787 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-03-18T15:31:25.818 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-03-18T15:31:25.828 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking postgresql-client (14+238) ... 2024-03-18T15:31:26.072 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package postgresql-contrib. 2024-03-18T15:31:26.089 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-03-18T15:31:26.130 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking postgresql-contrib (14+238) ... 2024-03-18T15:31:26.194 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-18T15:31:26.304 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-03-18T15:31:26.433 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package tcl8.6. 2024-03-18T15:31:26.462 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-03-18T15:31:26.499 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-03-18T15:31:26.555 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-03-18T15:31:26.752 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package tcl. 2024-03-18T15:31:26.781 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-03-18T15:31:26.798 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-03-18T15:31:26.824 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-03-18T15:31:27.107 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up zip (3.0-12build2) ... 2024-03-18T15:31:27.172 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package wpasupplicant. 2024-03-18T15:31:27.201 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-03-18T15:31:27.233 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-03-18T15:31:27.238 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-03-18T15:31:27.592 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-03-18T15:31:27.610 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-03-18T15:31:27.650 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-03-18T15:31:27.878 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-03-18T15:31:27.954 INFO:teuthology.orchestra.run.smithi005.stdout:Selecting previously unselected package usb-modeswitch. 2024-03-18T15:31:27.983 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-03-18T15:31:28.021 INFO:teuthology.orchestra.run.smithi005.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-03-18T15:31:28.137 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-03-18T15:31:28.263 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-03-18T15:31:28.326 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up postgresql-client-common (238) ... 2024-03-18T15:31:28.372 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-18T15:31:28.535 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up flex (2.6.4-8build2) ... 2024-03-18T15:31:28.648 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-03-18T15:31:28.669 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-03-18T15:31:28.795 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-03-18T15:31:28.921 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-03-18T15:31:29.047 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-03-18T15:31:29.148 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-18T15:31:29.341 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-03-18T15:31:29.600 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-03-18T15:31:29.689 INFO:teuthology.orchestra.run.smithi005.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-03-18T15:31:29.762 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-03-18T15:31:29.977 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-18T15:31:30.158 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-03-18T15:31:30.158 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-03-18T15:31:30.261 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-03-18T15:31:30.546 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-18T15:31:30.657 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-03-18T15:31:30.766 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-18T15:31:30.872 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-03-18T15:31:30.892 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-03-18T15:31:31.021 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-03-18T15:31:31.124 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-03-18T15:31:31.147 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-03-18T15:31:31.264 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-03-18T15:31:31.266 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-ceph-common (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:31.374 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-03-18T15:31:31.568 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-idna (3.3-1) ... 2024-03-18T15:31:31.877 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-18T15:31:32.013 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-03-18T15:31:32.139 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up dump (0.4b47-3) ... 2024-03-18T15:31:32.224 INFO:teuthology.orchestra.run.smithi005.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-03-18T15:31:32.291 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-03-18T15:31:32.392 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-03-18T15:31:32.651 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up tcl (8.6.11+1build2) ... 2024-03-18T15:31:33.051 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-03-18T15:31:33.294 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up libjson-perl (4.04000-1) ... 2024-03-18T15:31:33.311 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-18T15:31:33.404 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-03-18T15:31:33.436 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-toml (0.10.2-1) ... 2024-03-18T15:31:33.502 INFO:teuthology.orchestra.run.smithi005.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-03-18T15:31:33.597 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-03-18T15:31:33.671 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-18T15:31:33.723 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-03-18T15:31:33.789 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-03-18T15:31:33.899 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-03-18T15:31:33.915 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-18T15:31:34.025 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-03-18T15:31:34.151 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-03-18T15:31:34.216 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-03-18T15:31:34.319 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up postgresql-client (14+238) ... 2024-03-18T15:31:34.428 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-03-18T15:31:34.484 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-03-18T15:31:34.735 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-18T15:31:34.861 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-03-18T15:31:34.867 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:31:34.867 INFO:teuthology.orchestra.run.smithi005.stdout:The following network interfaces were found in /etc/network/interfaces 2024-03-18T15:31:34.867 INFO:teuthology.orchestra.run.smithi005.stdout:which means they are currently configured by ifupdown: 2024-03-18T15:31:34.867 INFO:teuthology.orchestra.run.smithi005.stdout:- ens1f0 2024-03-18T15:31:34.867 INFO:teuthology.orchestra.run.smithi005.stdout:If you want to manage those interfaces with NetworkManager instead 2024-03-18T15:31:34.868 INFO:teuthology.orchestra.run.smithi005.stdout:remove their configuration from /etc/network/interfaces. 2024-03-18T15:31:34.868 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:31:35.002 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-03-18T15:31:35.162 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-03-18T15:31:35.337 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-03-18T15:31:35.413 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-03-18T15:31:35.673 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-03-18T15:31:35.681 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-03-18T15:31:35.932 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-03-18T15:31:36.175 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-03-18T15:31:36.198 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-03-18T15:31:36.341 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-03-18T15:31:36.434 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-18T15:31:36.467 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-03-18T15:31:36.560 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-03-18T15:31:36.593 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up postgresql-common (238) ... 2024-03-18T15:31:36.870 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-18T15:31:36.971 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-18T15:31:37.247 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-03-18T15:31:37.526 INFO:teuthology.orchestra.run.smithi005.stdout:Adding user postgres to group ssl-cert 2024-03-18T15:31:37.565 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-03-18T15:31:37.735 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:31:37.735 INFO:teuthology.orchestra.run.smithi005.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-03-18T15:31:37.802 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-18T15:31:37.925 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-03-18T15:31:37.926 INFO:teuthology.orchestra.run.smithi005.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-03-18T15:31:37.926 INFO:teuthology.orchestra.run.smithi005.stdout:Removing obsolete dictionary files: 2024-03-18T15:31:38.052 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-03-18T15:31:38.140 INFO:teuthology.orchestra.run.smithi080.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-03-18T15:31:38.378 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-18T15:31:38.479 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-03-18T15:31:38.509 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-03-18T15:31:38.856 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-03-18T15:31:38.894 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-18T15:31:39.124 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up lua-any (27ubuntu1) ... 2024-03-18T15:31:39.241 INFO:teuthology.orchestra.run.smithi005.stdout:Creating new PostgreSQL cluster 14/main ... 2024-03-18T15:31:39.242 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-portend (3.0.0-1) ... 2024-03-18T15:31:39.258 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-03-18T15:31:39.301 INFO:teuthology.orchestra.run.smithi005.stdout:The files belonging to this database system will be owned by user "postgres". 2024-03-18T15:31:39.301 INFO:teuthology.orchestra.run.smithi005.stdout:This user must also own the server process. 2024-03-18T15:31:39.301 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:31:39.301 INFO:teuthology.orchestra.run.smithi005.stdout:The database cluster will be initialized with locale "en_US". 2024-03-18T15:31:39.302 INFO:teuthology.orchestra.run.smithi005.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-03-18T15:31:39.302 INFO:teuthology.orchestra.run.smithi005.stdout:The default text search configuration will be set to "english". 2024-03-18T15:31:39.302 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:31:39.302 INFO:teuthology.orchestra.run.smithi005.stdout:Data page checksums are disabled. 2024-03-18T15:31:39.302 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:31:39.302 INFO:teuthology.orchestra.run.smithi005.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-03-18T15:31:39.302 INFO:teuthology.orchestra.run.smithi005.stdout:creating subdirectories ... ok 2024-03-18T15:31:39.302 INFO:teuthology.orchestra.run.smithi005.stdout:selecting dynamic shared memory implementation ... posix 2024-03-18T15:31:39.314 INFO:teuthology.orchestra.run.smithi005.stdout:selecting default max_connections ... 100 2024-03-18T15:31:39.332 INFO:teuthology.orchestra.run.smithi005.stdout:selecting default shared_buffers ... 128MB 2024-03-18T15:31:39.340 INFO:teuthology.orchestra.run.smithi005.stdout:selecting default time zone ... Etc/UTC 2024-03-18T15:31:39.341 INFO:teuthology.orchestra.run.smithi005.stdout:creating configuration files ... ok 2024-03-18T15:31:39.459 INFO:teuthology.orchestra.run.smithi005.stdout:running bootstrap script ... ok 2024-03-18T15:31:39.492 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-18T15:31:39.619 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-18T15:31:39.928 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-03-18T15:31:40.058 INFO:teuthology.orchestra.run.smithi005.stdout:performing post-bootstrap initialization ... ok 2024-03-18T15:31:40.221 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-18T15:31:40.488 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-03-18T15:31:40.664 INFO:teuthology.orchestra.run.smithi005.stdout:syncing data to disk ... ok 2024-03-18T15:31:40.700 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-rsa (4.8-1) ... 2024-03-18T15:31:40.968 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-03-18T15:31:41.042 INFO:teuthology.orchestra.run.smithi005.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-03-18T15:31:41.228 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-18T15:31:41.479 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-03-18T15:31:41.848 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-03-18T15:31:42.133 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-03-18T15:31:42.460 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-18T15:31:42.586 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up librados2 (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:42.712 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libsqlite3-mod-ceph (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:42.863 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-03-18T15:31:43.183 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libcephfs2 (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:43.301 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libradosstriper1 (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:43.427 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-03-18T15:31:43.712 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up librbd1 (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:43.793 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-03-18T15:31:43.838 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-mgr-modules-core (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:43.919 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up postgresql-contrib (14+238) ... 2024-03-18T15:31:43.980 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-fuse (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:44.045 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up postgresql (14+238) ... 2024-03-18T15:31:44.171 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-03-18T15:31:44.190 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-18T15:31:44.191 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-18T15:31:44.314 INFO:teuthology.orchestra.run.smithi005.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-03-18T15:31:44.576 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-03-18T15:31:44.576 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-03-18T15:31:44.807 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libcephfs-dev (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:44.908 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-rados (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:45.034 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up librgw2 (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:45.158 INFO:teuthology.orchestra.run.smithi005.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-03-18T15:31:45.160 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up cephfs-top (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:45.286 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-18T15:31:45.418 INFO:teuthology.orchestra.run.smithi005.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-18T15:31:45.553 INFO:teuthology.orchestra.run.smithi005.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-18T15:31:46.474 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-rbd (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:46.602 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up rbd-fuse (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:46.728 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-rgw (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:46.854 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-cephfs (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:46.882 INFO:teuthology.orchestra.run.smithi005.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-03-18T15:31:46.980 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up cephfs-shell (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:46.981 INFO:teuthology.orchestra.run.smithi005.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-18T15:31:47.106 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-common (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:47.462 INFO:teuthology.orchestra.run.smithi005.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-18T15:31:47.468 DEBUG:teuthology.parallel:result is None 2024-03-18T15:31:47.590 INFO:teuthology.orchestra.run.smithi080.stdout:Adding group ceph....done 2024-03-18T15:31:47.959 INFO:teuthology.orchestra.run.smithi080.stdout:Adding system user ceph....done 2024-03-18T15:31:48.127 INFO:teuthology.orchestra.run.smithi080.stdout:Setting system user ceph properties....done 2024-03-18T15:31:48.145 INFO:teuthology.orchestra.run.smithi080.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-03-18T15:31:48.288 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-03-18T15:31:48.576 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-03-18T15:31:49.082 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up cephfs-mirror (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:49.271 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-03-18T15:31:49.271 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-03-18T15:31:49.734 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-test (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:49.860 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up radosgw (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:50.324 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-18T15:31:50.324 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-18T15:31:50.762 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-base (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:51.110 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-03-18T15:31:51.589 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-mds (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:51.824 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-18T15:31:51.824 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-18T15:31:52.291 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-mgr (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:52.492 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-18T15:31:52.492 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-18T15:31:53.001 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-osd (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:53.266 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-18T15:31:53.267 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-18T15:31:53.736 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-mgr-k8sevents (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:53.862 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:54.005 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-mon (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:54.220 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-18T15:31:54.220 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-18T15:31:54.723 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-mgr-cephadm (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:54.865 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:54.991 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-mgr-dashboard (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:55.168 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-volume (19.0.0-2162-g3d78f5ba-1jammy) ... 2024-03-18T15:31:55.487 INFO:teuthology.orchestra.run.smithi080.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-03-18T15:31:55.588 INFO:teuthology.orchestra.run.smithi080.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-18T15:31:55.741 INFO:teuthology.orchestra.run.smithi080.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-18T15:31:56.951 INFO:teuthology.orchestra.run.smithi080.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-18T15:31:57.609 INFO:teuthology.orchestra.run.smithi080.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-18T15:31:57.612 DEBUG:teuthology.orchestra.run.smithi080:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-03-18T15:31:57.716 INFO:teuthology.orchestra.run.smithi080.stdout:Reading package lists... 2024-03-18T15:31:57.913 INFO:teuthology.orchestra.run.smithi080.stdout:Building dependency tree... 2024-03-18T15:31:57.914 INFO:teuthology.orchestra.run.smithi080.stdout:Reading state information... 2024-03-18T15:31:58.091 INFO:teuthology.orchestra.run.smithi080.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-03-18T15:31:58.091 INFO:teuthology.orchestra.run.smithi080.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-03-18T15:31:58.091 INFO:teuthology.orchestra.run.smithi080.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2024-03-18T15:31:58.091 INFO:teuthology.orchestra.run.smithi080.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2024-03-18T15:31:58.091 INFO:teuthology.orchestra.run.smithi080.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-03-18T15:31:58.091 INFO:teuthology.orchestra.run.smithi080.stdout:The following packages were automatically installed and are no longer required: 2024-03-18T15:31:58.092 INFO:teuthology.orchestra.run.smithi080.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-18T15:31:58.092 INFO:teuthology.orchestra.run.smithi080.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-18T15:31:58.092 INFO:teuthology.orchestra.run.smithi080.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-18T15:31:58.092 INFO:teuthology.orchestra.run.smithi080.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-18T15:31:58.092 INFO:teuthology.orchestra.run.smithi080.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-18T15:31:58.092 INFO:teuthology.orchestra.run.smithi080.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-18T15:31:58.093 INFO:teuthology.orchestra.run.smithi080.stdout:The following additional packages will be installed: 2024-03-18T15:31:58.093 INFO:teuthology.orchestra.run.smithi080.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-03-18T15:31:58.094 INFO:teuthology.orchestra.run.smithi080.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-03-18T15:31:58.094 INFO:teuthology.orchestra.run.smithi080.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2024-03-18T15:31:58.094 INFO:teuthology.orchestra.run.smithi080.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-03-18T15:31:58.094 INFO:teuthology.orchestra.run.smithi080.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-03-18T15:31:58.095 INFO:teuthology.orchestra.run.smithi080.stdout:Suggested packages: 2024-03-18T15:31:58.095 INFO:teuthology.orchestra.run.smithi080.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-03-18T15:31:58.095 INFO:teuthology.orchestra.run.smithi080.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2024-03-18T15:31:58.095 INFO:teuthology.orchestra.run.smithi080.stdout: libengine-pkcs11-openssl 2024-03-18T15:31:58.133 INFO:teuthology.orchestra.run.smithi080.stdout:The following NEW packages will be installed: 2024-03-18T15:31:58.134 INFO:teuthology.orchestra.run.smithi080.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-03-18T15:31:58.134 INFO:teuthology.orchestra.run.smithi080.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-03-18T15:31:58.134 INFO:teuthology.orchestra.run.smithi080.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-03-18T15:31:58.134 INFO:teuthology.orchestra.run.smithi080.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-03-18T15:31:58.134 INFO:teuthology.orchestra.run.smithi080.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-03-18T15:31:58.135 INFO:teuthology.orchestra.run.smithi080.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-03-18T15:31:58.135 INFO:teuthology.orchestra.run.smithi080.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-03-18T15:31:58.135 INFO:teuthology.orchestra.run.smithi080.stdout:The following packages will be upgraded: 2024-03-18T15:31:58.136 INFO:teuthology.orchestra.run.smithi080.stdout: libssl3 2024-03-18T15:31:58.318 INFO:teuthology.orchestra.run.smithi080.stdout:1 upgraded, 39 newly installed, 0 to remove and 284 not upgraded. 2024-03-18T15:31:58.318 INFO:teuthology.orchestra.run.smithi080.stdout:Need to get 54.6 MB of archives. 2024-03-18T15:31:58.318 INFO:teuthology.orchestra.run.smithi080.stdout:After this operation, 202 MB of additional disk space will be used. 2024-03-18T15:31:58.318 INFO:teuthology.orchestra.run.smithi080.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-03-18T15:31:58.802 INFO:teuthology.orchestra.run.smithi080.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-03-18T15:31:59.060 INFO:teuthology.orchestra.run.smithi080.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-03-18T15:31:59.094 INFO:teuthology.orchestra.run.smithi080.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-03-18T15:31:59.096 INFO:teuthology.orchestra.run.smithi080.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-03-18T15:31:59.102 INFO:teuthology.orchestra.run.smithi080.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-03-18T15:31:59.104 INFO:teuthology.orchestra.run.smithi080.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-03-18T15:31:59.105 INFO:teuthology.orchestra.run.smithi080.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-03-18T15:31:59.108 INFO:teuthology.orchestra.run.smithi080.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-03-18T15:31:59.111 INFO:teuthology.orchestra.run.smithi080.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-03-18T15:31:59.276 INFO:teuthology.orchestra.run.smithi080.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-03-18T15:31:59.276 INFO:teuthology.orchestra.run.smithi080.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-03-18T15:31:59.520 INFO:teuthology.orchestra.run.smithi080.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-03-18T15:32:00.705 INFO:teuthology.orchestra.run.smithi080.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-03-18T15:32:00.709 INFO:teuthology.orchestra.run.smithi080.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-03-18T15:32:00.710 INFO:teuthology.orchestra.run.smithi080.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-03-18T15:32:00.717 INFO:teuthology.orchestra.run.smithi080.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-03-18T15:32:00.717 INFO:teuthology.orchestra.run.smithi080.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-03-18T15:32:00.728 INFO:teuthology.orchestra.run.smithi080.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-03-18T15:32:00.731 INFO:teuthology.orchestra.run.smithi080.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-03-18T15:32:00.749 INFO:teuthology.orchestra.run.smithi080.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-03-18T15:32:00.913 INFO:teuthology.orchestra.run.smithi080.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-03-18T15:32:01.636 INFO:teuthology.orchestra.run.smithi080.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-03-18T15:32:01.637 INFO:teuthology.orchestra.run.smithi080.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-03-18T15:32:01.663 INFO:teuthology.orchestra.run.smithi080.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-03-18T15:32:01.741 INFO:teuthology.orchestra.run.smithi080.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-03-18T15:32:01.749 INFO:teuthology.orchestra.run.smithi080.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-03-18T15:32:01.750 INFO:teuthology.orchestra.run.smithi080.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-03-18T15:32:01.752 INFO:teuthology.orchestra.run.smithi080.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-03-18T15:32:01.753 INFO:teuthology.orchestra.run.smithi080.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-03-18T15:32:01.779 INFO:teuthology.orchestra.run.smithi080.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-03-18T15:32:01.782 INFO:teuthology.orchestra.run.smithi080.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-03-18T15:32:02.523 INFO:teuthology.orchestra.run.smithi080.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-03-18T15:32:02.605 INFO:teuthology.orchestra.run.smithi080.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-03-18T15:32:02.605 INFO:teuthology.orchestra.run.smithi080.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-03-18T15:32:02.606 INFO:teuthology.orchestra.run.smithi080.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-03-18T15:32:02.697 INFO:teuthology.orchestra.run.smithi080.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-03-18T15:32:02.711 INFO:teuthology.orchestra.run.smithi080.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-03-18T15:32:03.165 INFO:teuthology.orchestra.run.smithi080.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-03-18T15:32:03.168 INFO:teuthology.orchestra.run.smithi080.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-03-18T15:32:03.577 INFO:teuthology.orchestra.run.smithi080.stdout:Preconfiguring packages ... 2024-03-18T15:32:03.728 INFO:teuthology.orchestra.run.smithi080.stdout:Fetched 54.6 MB in 5s (10.9 MB/s) 2024-03-18T15:32:03.876 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package flex. 2024-03-18T15:32:03.977 INFO:teuthology.orchestra.run.smithi080.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173660 files and directories currently installed.) 2024-03-18T15:32:03.983 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-03-18T15:32:04.024 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking flex (2.6.4-8build2) ... 2024-03-18T15:32:04.574 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-03-18T15:32:04.764 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-03-18T15:32:05.183 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-18T15:32:05.699 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package bison. 2024-03-18T15:32:05.802 INFO:teuthology.orchestra.run.smithi080.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173733 files and directories currently installed.) 2024-03-18T15:32:05.808 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-03-18T15:32:05.859 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-03-18T15:32:06.252 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-03-18T15:32:06.280 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-03-18T15:32:06.318 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-03-18T15:32:06.655 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package dump. 2024-03-18T15:32:06.683 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-03-18T15:32:06.722 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking dump (0.4b47-3) ... 2024-03-18T15:32:07.001 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package indent. 2024-03-18T15:32:07.028 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-03-18T15:32:07.059 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-03-18T15:32:07.320 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-03-18T15:32:07.336 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-03-18T15:32:07.496 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-03-18T15:32:07.826 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-03-18T15:32:07.853 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-03-18T15:32:07.892 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-03-18T15:32:08.153 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-03-18T15:32:08.169 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-03-18T15:32:08.211 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-03-18T15:32:08.548 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libfl2:amd64. 2024-03-18T15:32:08.576 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-03-18T15:32:08.615 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-03-18T15:32:08.834 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-03-18T15:32:08.850 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-03-18T15:32:08.892 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-03-18T15:32:09.145 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libjson-perl. 2024-03-18T15:32:09.162 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-03-18T15:32:09.203 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-03-18T15:32:09.515 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libllvm14:amd64. 2024-03-18T15:32:09.543 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-03-18T15:32:09.581 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-03-18T15:32:11.301 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-03-18T15:32:11.329 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-03-18T15:32:11.368 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-03-18T15:32:11.612 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libmbim-proxy. 2024-03-18T15:32:11.628 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-03-18T15:32:11.662 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-03-18T15:32:11.981 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-03-18T15:32:12.009 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-03-18T15:32:12.048 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-03-18T15:32:12.393 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libndp0:amd64. 2024-03-18T15:32:12.420 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-03-18T15:32:12.459 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-03-18T15:32:12.797 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libnm0:amd64. 2024-03-18T15:32:12.824 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-03-18T15:32:12.863 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-03-18T15:32:13.217 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libpq5:amd64. 2024-03-18T15:32:13.244 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-18T15:32:13.283 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-03-18T15:32:13.636 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-03-18T15:32:13.664 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-03-18T15:32:13.702 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-03-18T15:32:13.972 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libqmi-proxy. 2024-03-18T15:32:13.988 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-03-18T15:32:14.022 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-03-18T15:32:14.283 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-03-18T15:32:14.311 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-03-18T15:32:14.349 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-18T15:32:14.828 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-03-18T15:32:14.856 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-03-18T15:32:14.886 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-03-18T15:32:15.198 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package modemmanager. 2024-03-18T15:32:15.225 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-03-18T15:32:15.264 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-03-18T15:32:16.338 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package network-manager. 2024-03-18T15:32:16.365 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-03-18T15:32:16.412 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-03-18T15:32:17.004 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ppp. 2024-03-18T15:32:17.031 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-03-18T15:32:17.306 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-03-18T15:32:17.738 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package pptp-linux. 2024-03-18T15:32:17.764 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-03-18T15:32:17.804 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-03-18T15:32:18.013 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package network-manager-pptp. 2024-03-18T15:32:18.030 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-03-18T15:32:18.072 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-03-18T15:32:18.375 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package postgresql-client-common. 2024-03-18T15:32:18.403 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-03-18T15:32:18.442 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking postgresql-client-common (238) ... 2024-03-18T15:32:18.746 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package postgresql-client-14. 2024-03-18T15:32:18.774 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-18T15:32:18.812 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-18T15:32:19.368 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package postgresql-common. 2024-03-18T15:32:19.396 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-03-18T15:32:19.453 INFO:teuthology.orchestra.run.smithi080.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-03-18T15:32:19.482 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking postgresql-common (238) ... 2024-03-18T15:32:19.951 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package postgresql-14. 2024-03-18T15:32:20.004 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-18T15:32:20.021 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-18T15:32:21.321 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package postgresql. 2024-03-18T15:32:21.338 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-03-18T15:32:21.379 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking postgresql (14+238) ... 2024-03-18T15:32:21.607 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package postgresql-client. 2024-03-18T15:32:21.623 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-03-18T15:32:21.657 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking postgresql-client (14+238) ... 2024-03-18T15:32:21.909 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package postgresql-contrib. 2024-03-18T15:32:21.926 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-03-18T15:32:21.968 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking postgresql-contrib (14+238) ... 2024-03-18T15:32:22.270 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package tcl8.6. 2024-03-18T15:32:22.299 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-03-18T15:32:22.337 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-03-18T15:32:22.590 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package tcl. 2024-03-18T15:32:22.618 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-03-18T15:32:22.660 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-03-18T15:32:23.035 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package wpasupplicant. 2024-03-18T15:32:23.064 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-03-18T15:32:23.101 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-03-18T15:32:23.481 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-03-18T15:32:23.509 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-03-18T15:32:23.547 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-03-18T15:32:23.909 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package usb-modeswitch. 2024-03-18T15:32:23.937 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-03-18T15:32:23.976 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-03-18T15:32:24.272 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up postgresql-client-common (238) ... 2024-03-18T15:32:24.482 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up flex (2.6.4-8build2) ... 2024-03-18T15:32:24.609 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-03-18T15:32:24.735 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-03-18T15:32:24.861 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-03-18T15:32:24.987 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-03-18T15:32:25.088 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-18T15:32:25.574 INFO:teuthology.orchestra.run.smithi080.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-03-18T15:32:25.653 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-03-18T15:32:26.016 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-03-18T15:32:26.016 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-03-18T15:32:26.466 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-03-18T15:32:26.600 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-18T15:32:26.726 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-03-18T15:32:26.852 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-03-18T15:32:26.978 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-03-18T15:32:27.104 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-03-18T15:32:27.213 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-03-18T15:32:28.028 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-03-18T15:32:28.404 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up dump (0.4b47-3) ... 2024-03-18T15:32:28.498 INFO:teuthology.orchestra.run.smithi080.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-03-18T15:32:28.547 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-03-18T15:32:28.673 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-03-18T15:32:28.800 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up tcl (8.6.11+1build2) ... 2024-03-18T15:32:28.959 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libjson-perl (4.04000-1) ... 2024-03-18T15:32:29.085 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-03-18T15:32:29.175 INFO:teuthology.orchestra.run.smithi080.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-03-18T15:32:29.245 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-03-18T15:32:29.363 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-03-18T15:32:29.514 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-03-18T15:32:29.640 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-03-18T15:32:29.775 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-03-18T15:32:29.943 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up postgresql-client (14+238) ... 2024-03-18T15:32:30.069 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-03-18T15:32:30.552 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-03-18T15:32:30.552 INFO:teuthology.orchestra.run.smithi080.stdout:The following network interfaces were found in /etc/network/interfaces 2024-03-18T15:32:30.552 INFO:teuthology.orchestra.run.smithi080.stdout:which means they are currently configured by ifupdown: 2024-03-18T15:32:30.552 INFO:teuthology.orchestra.run.smithi080.stdout:- enp3s0f1 2024-03-18T15:32:30.552 INFO:teuthology.orchestra.run.smithi080.stdout:If you want to manage those interfaces with NetworkManager instead 2024-03-18T15:32:30.552 INFO:teuthology.orchestra.run.smithi080.stdout:remove their configuration from /etc/network/interfaces. 2024-03-18T15:32:30.552 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-03-18T15:32:30.669 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-03-18T15:32:31.013 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-03-18T15:32:31.361 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-03-18T15:32:31.880 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-03-18T15:32:32.065 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-03-18T15:32:32.191 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-03-18T15:32:32.317 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up postgresql-common (238) ... 2024-03-18T15:32:33.202 INFO:teuthology.orchestra.run.smithi080.stdout:Adding user postgres to group ssl-cert 2024-03-18T15:32:33.455 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-03-18T15:32:33.455 INFO:teuthology.orchestra.run.smithi080.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-03-18T15:32:33.666 INFO:teuthology.orchestra.run.smithi080.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-03-18T15:32:33.666 INFO:teuthology.orchestra.run.smithi080.stdout:Removing obsolete dictionary files: 2024-03-18T15:32:34.159 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-03-18T15:32:34.538 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-18T15:32:34.864 INFO:teuthology.orchestra.run.smithi080.stdout:Creating new PostgreSQL cluster 14/main ... 2024-03-18T15:32:34.876 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-03-18T15:32:34.916 INFO:teuthology.orchestra.run.smithi080.stdout:The files belonging to this database system will be owned by user "postgres". 2024-03-18T15:32:34.916 INFO:teuthology.orchestra.run.smithi080.stdout:This user must also own the server process. 2024-03-18T15:32:34.916 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-03-18T15:32:34.916 INFO:teuthology.orchestra.run.smithi080.stdout:The database cluster will be initialized with locale "en_US". 2024-03-18T15:32:34.916 INFO:teuthology.orchestra.run.smithi080.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-03-18T15:32:34.917 INFO:teuthology.orchestra.run.smithi080.stdout:The default text search configuration will be set to "english". 2024-03-18T15:32:34.917 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-03-18T15:32:34.917 INFO:teuthology.orchestra.run.smithi080.stdout:Data page checksums are disabled. 2024-03-18T15:32:34.917 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-03-18T15:32:34.917 INFO:teuthology.orchestra.run.smithi080.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-03-18T15:32:34.917 INFO:teuthology.orchestra.run.smithi080.stdout:creating subdirectories ... ok 2024-03-18T15:32:34.917 INFO:teuthology.orchestra.run.smithi080.stdout:selecting dynamic shared memory implementation ... posix 2024-03-18T15:32:34.931 INFO:teuthology.orchestra.run.smithi080.stdout:selecting default max_connections ... 100 2024-03-18T15:32:34.950 INFO:teuthology.orchestra.run.smithi080.stdout:selecting default shared_buffers ... 128MB 2024-03-18T15:32:34.960 INFO:teuthology.orchestra.run.smithi080.stdout:selecting default time zone ... Etc/UTC 2024-03-18T15:32:34.961 INFO:teuthology.orchestra.run.smithi080.stdout:creating configuration files ... ok 2024-03-18T15:32:35.075 INFO:teuthology.orchestra.run.smithi080.stdout:running bootstrap script ... ok 2024-03-18T15:32:35.709 INFO:teuthology.orchestra.run.smithi080.stdout:performing post-bootstrap initialization ... ok 2024-03-18T15:32:36.366 INFO:teuthology.orchestra.run.smithi080.stdout:syncing data to disk ... ok 2024-03-18T15:32:36.751 INFO:teuthology.orchestra.run.smithi080.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-03-18T15:32:39.510 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-03-18T15:32:39.636 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up postgresql-contrib (14+238) ... 2024-03-18T15:32:39.754 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up postgresql (14+238) ... 2024-03-18T15:32:39.872 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-03-18T15:32:39.981 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-03-18T15:32:40.235 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-03-18T15:32:40.235 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-03-18T15:32:40.737 INFO:teuthology.orchestra.run.smithi080.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-03-18T15:32:41.012 INFO:teuthology.orchestra.run.smithi080.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-18T15:32:41.179 INFO:teuthology.orchestra.run.smithi080.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-18T15:32:42.533 INFO:teuthology.orchestra.run.smithi080.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-03-18T15:32:42.641 INFO:teuthology.orchestra.run.smithi080.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-18T15:32:43.125 INFO:teuthology.orchestra.run.smithi080.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-18T15:32:43.130 DEBUG:teuthology.parallel:result is None 2024-03-18T15:32:43.131 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-18T15:32:43.275 DEBUG:teuthology.orchestra.run.smithi005:> dpkg-query -W -f '${Version}' ceph 2024-03-18T15:32:43.295 INFO:teuthology.orchestra.run.smithi005.stdout:19.0.0-2162-g3d78f5ba-1jammy 2024-03-18T15:32:43.296 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2162-g3d78f5ba-1jammy 2024-03-18T15:32:43.296 INFO:teuthology.task.install:The correct ceph version 19.0.0-2162-g3d78f5ba-1jammy is installed. 2024-03-18T15:32:43.298 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=3d78f5ba07bf18880172c07d4d7b4ac00c9d30af 2024-03-18T15:32:43.530 DEBUG:teuthology.orchestra.run.smithi080:> dpkg-query -W -f '${Version}' ceph 2024-03-18T15:32:43.549 INFO:teuthology.orchestra.run.smithi080.stdout:19.0.0-2162-g3d78f5ba-1jammy 2024-03-18T15:32:43.550 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2162-g3d78f5ba-1jammy 2024-03-18T15:32:43.550 INFO:teuthology.task.install:The correct ceph version 19.0.0-2162-g3d78f5ba-1jammy is installed. 2024-03-18T15:32:43.551 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-18T15:32:43.552 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-03-18T15:32:43.552 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-18T15:32:43.567 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-03-18T15:32:43.567 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-18T15:32:43.606 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-18T15:32:43.606 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-03-18T15:32:43.606 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/daemon-helper 2024-03-18T15:32:43.622 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-18T15:32:43.678 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-03-18T15:32:43.678 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/usr/bin/daemon-helper 2024-03-18T15:32:43.691 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-18T15:32:43.745 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-18T15:32:43.745 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-03-18T15:32:43.745 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-18T15:32:43.758 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-18T15:32:43.813 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-03-18T15:32:43.813 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-18T15:32:43.827 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-18T15:32:43.881 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-18T15:32:43.882 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-03-18T15:32:43.882 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/stdin-killer 2024-03-18T15:32:43.895 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-18T15:32:43.949 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-03-18T15:32:43.949 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/usr/bin/stdin-killer 2024-03-18T15:32:43.963 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-18T15:32:44.018 INFO:teuthology.run_tasks:Running task ceph... 2024-03-18T15:32:44.109 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-03-18T15:32:44.110 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 777 /var/log/ceph 2024-03-18T15:32:44.112 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod 777 /var/log/ceph 2024-03-18T15:32:44.125 INFO:tasks.ceph:Disabling ceph logrotate... 2024-03-18T15:32:44.125 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-18T15:32:44.167 DEBUG:teuthology.orchestra.run.smithi080:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-18T15:32:44.181 INFO:tasks.ceph:Creating extra log directories... 2024-03-18T15:32:44.181 DEBUG:teuthology.orchestra.run.smithi005:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-18T15:32:44.223 DEBUG:teuthology.orchestra.run.smithi080:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-18T15:32:44.243 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-03-18T15:32:44.243 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-03-18T15:32:44.244 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-03-18T15:32:44.244 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-03-18T15:32:44.244 DEBUG:teuthology.orchestra.run.smithi005:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-18T15:32:44.282 DEBUG:teuthology.orchestra.run.smithi005:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-18T15:32:44.359 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-03-18T15:32:44.359 DEBUG:teuthology.orchestra.run.smithi080:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-18T15:32:44.365 DEBUG:teuthology.orchestra.run.smithi080:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-18T15:32:44.447 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-03-18T15:32:44.448 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'mons down, quorum', 'out of quorum', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-03-18T15:32:44.448 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608878', 'branch': 'wip-batrick-testing-20240314.183243', 'description': 'fs/shell/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/ubuntu_latest mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/cephfs-shell}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7608878', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-batrick-testing-20240314.183243'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'mons down, quorum', 'out of quorum', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN'], 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-batrick-testing-20240314.183243', 'sha1': '4e19f704afd9957ea657ab858f9ecf76401fef6f'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 75, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7']], 'seed': 4770, 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af', 'sleep_before_teardown': 0, 'subset': '498/512', 'suite': 'fs', 'suite_branch': 'wip-batrick-testing-20240314.183243', 'suite_path': '/home/teuthworker/src/github.com_batrick_ceph_4e19f704afd9957ea657ab858f9ecf76401fef6f/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/batrick/ceph', 'suite_sha1': '4e19f704afd9957ea657ab858f9ecf76401fef6f', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi005.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBeylqcswqUr05yIIhATKJxEzhYbd6C3wDv2PdDWT4RmCi7iiC/LBqC9AtG2GTM3aparYmIWuzcG5frRFfBO4DI=', 'smithi080.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHNmff7IMarXwxiL+qz9KB3L6ZwbgMuqx413Lxw7UxAhVWj+9yFYOxhyKc9j8ckv7x6x9TGTIXs7Jwujqc59FpQ='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'mons down, quorum', 'out of quorum', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN'], 'sha1': '3d78f5ba07bf18880172c07d4d7b4ac00c9d30af', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_cephfs_shell']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e691533f9cbb33d85b2187bba20d7102f098636d', 'timestamp': '2024-03-18_14:19:46', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-03-18T15:32:44.448 DEBUG:teuthology.orchestra.run.smithi005:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-18T15:32:44.450 DEBUG:teuthology.orchestra.run.smithi080:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-18T15:32:44.491 DEBUG:teuthology.orchestra.run.smithi005:> sudo install -d -m0777 -- /var/run/ceph 2024-03-18T15:32:44.499 DEBUG:teuthology.orchestra.run.smithi080:> sudo install -d -m0777 -- /var/run/ceph 2024-03-18T15:32:44.546 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-03-18T15:32:44.546 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/scratch_devs of=/dev/stdout 2024-03-18T15:32:44.559 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-18T15:32:44.559 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_1 2024-03-18T15:32:44.607 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-18T15:32:44.607 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T15:32:44.607 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 873 Links: 1 2024-03-18T15:32:44.608 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T15:32:44.608 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-03-18 15:25:49.816481996 +0000 2024-03-18T15:32:44.608 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-03-18 15:25:49.544475719 +0000 2024-03-18T15:32:44.608 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-03-18 15:25:49.544475719 +0000 2024-03-18T15:32:44.608 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: - 2024-03-18T15:32:44.608 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-18T15:32:44.661 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-03-18T15:32:44.661 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-03-18T15:32:44.661 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000253713 s, 2.0 MB/s 2024-03-18T15:32:44.662 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-18T15:32:44.712 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_2 2024-03-18T15:32:44.758 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-18T15:32:44.758 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T15:32:44.759 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 898 Links: 1 2024-03-18T15:32:44.759 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T15:32:44.759 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-03-18 15:25:50.064487658 +0000 2024-03-18T15:32:44.759 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-03-18 15:25:50.060487569 +0000 2024-03-18T15:32:44.759 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-03-18 15:25:50.060487569 +0000 2024-03-18T15:32:44.759 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: - 2024-03-18T15:32:44.759 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-18T15:32:44.813 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-03-18T15:32:44.813 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-03-18T15:32:44.813 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000266059 s, 1.9 MB/s 2024-03-18T15:32:44.814 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-18T15:32:44.863 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_3 2024-03-18T15:32:44.910 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-18T15:32:44.910 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T15:32:44.910 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 913 Links: 1 2024-03-18T15:32:44.910 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T15:32:44.910 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-03-18 15:25:50.564498698 +0000 2024-03-18T15:32:44.910 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-03-18 15:25:50.560498609 +0000 2024-03-18T15:32:44.911 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-03-18 15:25:50.560498609 +0000 2024-03-18T15:32:44.911 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: - 2024-03-18T15:32:44.911 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-18T15:32:44.964 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-03-18T15:32:44.964 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-03-18T15:32:44.964 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000297727 s, 1.7 MB/s 2024-03-18T15:32:44.966 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-18T15:32:45.015 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_4 2024-03-18T15:32:45.062 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-18T15:32:45.062 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T15:32:45.062 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 926 Links: 1 2024-03-18T15:32:45.062 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T15:32:45.062 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-03-18 15:25:51.292514481 +0000 2024-03-18T15:32:45.063 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-03-18 15:25:51.028508916 +0000 2024-03-18T15:32:45.063 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-03-18 15:25:51.028508916 +0000 2024-03-18T15:32:45.063 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: - 2024-03-18T15:32:45.063 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-18T15:32:45.117 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-03-18T15:32:45.117 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-03-18T15:32:45.117 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000302606 s, 1.7 MB/s 2024-03-18T15:32:45.118 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-18T15:32:45.167 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-18T15:32:45.168 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-03-18T15:32:45.168 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/scratch_devs of=/dev/stdout 2024-03-18T15:32:45.174 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-18T15:32:45.174 DEBUG:teuthology.orchestra.run.smithi080:> stat /dev/vg_nvme/lv_1 2024-03-18T15:32:45.223 INFO:teuthology.orchestra.run.smithi080.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-18T15:32:45.223 INFO:teuthology.orchestra.run.smithi080.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T15:32:45.223 INFO:teuthology.orchestra.run.smithi080.stdout:Device: 5h/5d Inode: 875 Links: 1 2024-03-18T15:32:45.223 INFO:teuthology.orchestra.run.smithi080.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T15:32:45.223 INFO:teuthology.orchestra.run.smithi080.stdout:Access: 2024-03-18 15:25:46.114430644 +0000 2024-03-18T15:32:45.223 INFO:teuthology.orchestra.run.smithi080.stdout:Modify: 2024-03-18 15:25:45.858432927 +0000 2024-03-18T15:32:45.223 INFO:teuthology.orchestra.run.smithi080.stdout:Change: 2024-03-18 15:25:45.858432927 +0000 2024-03-18T15:32:45.223 INFO:teuthology.orchestra.run.smithi080.stdout: Birth: - 2024-03-18T15:32:45.224 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-18T15:32:45.278 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records in 2024-03-18T15:32:45.278 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records out 2024-03-18T15:32:45.278 INFO:teuthology.orchestra.run.smithi080.stderr:512 bytes copied, 0.000443436 s, 1.2 MB/s 2024-03-18T15:32:45.279 DEBUG:teuthology.orchestra.run.smithi080:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-18T15:32:45.328 DEBUG:teuthology.orchestra.run.smithi080:> stat /dev/vg_nvme/lv_2 2024-03-18T15:32:45.374 INFO:teuthology.orchestra.run.smithi080.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-18T15:32:45.375 INFO:teuthology.orchestra.run.smithi080.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T15:32:45.375 INFO:teuthology.orchestra.run.smithi080.stdout:Device: 5h/5d Inode: 892 Links: 1 2024-03-18T15:32:45.375 INFO:teuthology.orchestra.run.smithi080.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T15:32:45.375 INFO:teuthology.orchestra.run.smithi080.stdout:Access: 2024-03-18 15:25:46.614426184 +0000 2024-03-18T15:32:45.375 INFO:teuthology.orchestra.run.smithi080.stdout:Modify: 2024-03-18 15:25:46.358428466 +0000 2024-03-18T15:32:45.375 INFO:teuthology.orchestra.run.smithi080.stdout:Change: 2024-03-18 15:25:46.358428466 +0000 2024-03-18T15:32:45.375 INFO:teuthology.orchestra.run.smithi080.stdout: Birth: - 2024-03-18T15:32:45.375 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-18T15:32:45.430 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records in 2024-03-18T15:32:45.430 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records out 2024-03-18T15:32:45.431 INFO:teuthology.orchestra.run.smithi080.stderr:512 bytes copied, 0.000313975 s, 1.6 MB/s 2024-03-18T15:32:45.432 DEBUG:teuthology.orchestra.run.smithi080:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-18T15:32:45.482 DEBUG:teuthology.orchestra.run.smithi080:> stat /dev/vg_nvme/lv_3 2024-03-18T15:32:45.531 INFO:teuthology.orchestra.run.smithi080.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-18T15:32:45.531 INFO:teuthology.orchestra.run.smithi080.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T15:32:45.531 INFO:teuthology.orchestra.run.smithi080.stdout:Device: 5h/5d Inode: 911 Links: 1 2024-03-18T15:32:45.531 INFO:teuthology.orchestra.run.smithi080.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T15:32:45.531 INFO:teuthology.orchestra.run.smithi080.stdout:Access: 2024-03-18 15:25:47.102421829 +0000 2024-03-18T15:32:45.531 INFO:teuthology.orchestra.run.smithi080.stdout:Modify: 2024-03-18 15:25:46.850424078 +0000 2024-03-18T15:32:45.531 INFO:teuthology.orchestra.run.smithi080.stdout:Change: 2024-03-18 15:25:46.850424078 +0000 2024-03-18T15:32:45.531 INFO:teuthology.orchestra.run.smithi080.stdout: Birth: - 2024-03-18T15:32:45.532 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-18T15:32:45.585 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records in 2024-03-18T15:32:45.585 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records out 2024-03-18T15:32:45.586 INFO:teuthology.orchestra.run.smithi080.stderr:512 bytes copied, 0.0004177 s, 1.2 MB/s 2024-03-18T15:32:45.587 DEBUG:teuthology.orchestra.run.smithi080:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-18T15:32:45.636 DEBUG:teuthology.orchestra.run.smithi080:> stat /dev/vg_nvme/lv_4 2024-03-18T15:32:45.682 INFO:teuthology.orchestra.run.smithi080.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-18T15:32:45.683 INFO:teuthology.orchestra.run.smithi080.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-18T15:32:45.683 INFO:teuthology.orchestra.run.smithi080.stdout:Device: 5h/5d Inode: 920 Links: 1 2024-03-18T15:32:45.683 INFO:teuthology.orchestra.run.smithi080.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-18T15:32:45.683 INFO:teuthology.orchestra.run.smithi080.stdout:Access: 2024-03-18 15:25:47.586417513 +0000 2024-03-18T15:32:45.683 INFO:teuthology.orchestra.run.smithi080.stdout:Modify: 2024-03-18 15:25:47.334419760 +0000 2024-03-18T15:32:45.683 INFO:teuthology.orchestra.run.smithi080.stdout:Change: 2024-03-18 15:25:47.334419760 +0000 2024-03-18T15:32:45.683 INFO:teuthology.orchestra.run.smithi080.stdout: Birth: - 2024-03-18T15:32:45.683 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-18T15:32:45.738 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records in 2024-03-18T15:32:45.738 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records out 2024-03-18T15:32:45.738 INFO:teuthology.orchestra.run.smithi080.stderr:512 bytes copied, 0.000376854 s, 1.4 MB/s 2024-03-18T15:32:45.739 DEBUG:teuthology.orchestra.run.smithi080:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-18T15:32:45.788 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-18T15:32:45.788 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi005.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi080.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-18T15:32:45.788 INFO:tasks.ceph:Generating config... 2024-03-18T15:32:45.789 INFO:tasks.ceph:[client] client mount timeout = 600 2024-03-18T15:32:45.790 INFO:tasks.ceph:[client] debug client = 20 2024-03-18T15:32:45.790 INFO:tasks.ceph:[client] debug ms = 1 2024-03-18T15:32:45.790 INFO:tasks.ceph:[client] pid file = 2024-03-18T15:32:45.790 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-03-18T15:32:45.790 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-03-18T15:32:45.790 INFO:tasks.ceph:[mds] debug mds = 20 2024-03-18T15:32:45.790 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-03-18T15:32:45.790 INFO:tasks.ceph:[mds] debug ms = 1 2024-03-18T15:32:45.790 INFO:tasks.ceph:[mds] mds debug frag = True 2024-03-18T15:32:45.790 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-03-18T15:32:45.790 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-03-18T15:32:45.790 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-03-18T15:32:45.790 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-03-18T15:32:45.790 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-03-18T15:32:45.791 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-03-18T15:32:45.791 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-03-18T15:32:45.791 INFO:tasks.ceph:[mgr] debug ms = 1 2024-03-18T15:32:45.791 INFO:tasks.ceph:[mon] debug mon = 20 2024-03-18T15:32:45.791 INFO:tasks.ceph:[mon] debug ms = 1 2024-03-18T15:32:45.791 INFO:tasks.ceph:[mon] debug paxos = 20 2024-03-18T15:32:45.791 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-03-18T15:32:45.791 INFO:tasks.ceph:[osd] bdev async discard = True 2024-03-18T15:32:45.791 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-03-18T15:32:45.791 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-03-18T15:32:45.791 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-03-18T15:32:45.791 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-03-18T15:32:45.791 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-03-18T15:32:45.791 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-03-18T15:32:45.791 INFO:tasks.ceph:[osd] debug ms = 1 2024-03-18T15:32:45.792 INFO:tasks.ceph:[osd] debug osd = 20 2024-03-18T15:32:45.792 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-03-18T15:32:45.792 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-03-18T15:32:45.792 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-03-18T15:32:45.792 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-03-18T15:32:45.792 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-03-18T15:32:45.792 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-03-18T15:32:45.792 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-03-18T15:32:45.792 INFO:tasks.ceph:Setting up mon.a... 2024-03-18T15:32:45.792 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-03-18T15:32:45.827 INFO:teuthology.orchestra.run.smithi005.stdout:creating /etc/ceph/ceph.keyring 2024-03-18T15:32:45.836 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-03-18T15:32:45.912 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-18T15:32:45.965 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.5'), ('mon.b', '172.21.15.80'), ('mon.c', '[v2:172.21.15.80:3301,v1:172.21.15.80:6790]')] 2024-03-18T15:32:45.966 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.5,172.21.15.80,[v2:172.21.15.80:3301,v1:172.21.15.80:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.b': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-03-18T15:32:45.970 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-03-18T15:32:45.970 DEBUG:teuthology.orchestra.run.smithi005:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-03-18T15:32:46.018 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.5 --add b 172.21.15.80 --addv c '[v2:172.21.15.80:3301,v1:172.21.15.80:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-03-18T15:32:46.107 INFO:teuthology.orchestra.run.smithi005.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-03-18T15:32:46.108 INFO:teuthology.orchestra.run.smithi005.stdout:monmaptool: generated fsid ef54fe03-5aa7-4eb2-9b1e-66c9cc345b94 2024-03-18T15:32:46.108 INFO:teuthology.orchestra.run.smithi005.stdout:setting min_mon_release = quincy 2024-03-18T15:32:46.108 INFO:teuthology.orchestra.run.smithi005.stdout:epoch 0 2024-03-18T15:32:46.108 INFO:teuthology.orchestra.run.smithi005.stdout:fsid ef54fe03-5aa7-4eb2-9b1e-66c9cc345b94 2024-03-18T15:32:46.108 INFO:teuthology.orchestra.run.smithi005.stdout:last_changed 2024-03-18T15:32:46.108569+0000 2024-03-18T15:32:46.108 INFO:teuthology.orchestra.run.smithi005.stdout:created 2024-03-18T15:32:46.108569+0000 2024-03-18T15:32:46.108 INFO:teuthology.orchestra.run.smithi005.stdout:min_mon_release 17 (quincy) 2024-03-18T15:32:46.108 INFO:teuthology.orchestra.run.smithi005.stdout:election_strategy: 1 2024-03-18T15:32:46.108 INFO:teuthology.orchestra.run.smithi005.stdout:0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-03-18T15:32:46.108 INFO:teuthology.orchestra.run.smithi005.stdout:1: [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] mon.b 2024-03-18T15:32:46.108 INFO:teuthology.orchestra.run.smithi005.stdout:2: [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] mon.c 2024-03-18T15:32:46.108 INFO:teuthology.orchestra.run.smithi005.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-03-18T15:32:46.110 DEBUG:teuthology.orchestra.run.smithi005:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-03-18T15:32:46.159 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID ef54fe03-5aa7-4eb2-9b1e-66c9cc345b94... 2024-03-18T15:32:46.160 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-18T15:32:46.203 DEBUG:teuthology.orchestra.run.smithi080:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-18T15:32:46.233 INFO:teuthology.orchestra.run.smithi005.stdout:[global] 2024-03-18T15:32:46.233 INFO:teuthology.orchestra.run.smithi005.stdout: chdir = "" 2024-03-18T15:32:46.233 INFO:teuthology.orchestra.run.smithi005.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-18T15:32:46.233 INFO:teuthology.orchestra.run.smithi005.stdout: auth supported = cephx 2024-03-18T15:32:46.233 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:32:46.233 INFO:teuthology.orchestra.run.smithi005.stdout: filestore xattr use omap = true 2024-03-18T15:32:46.233 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:32:46.233 INFO:teuthology.orchestra.run.smithi005.stdout: mon clock drift allowed = 1.000 2024-03-18T15:32:46.233 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:32:46.233 INFO:teuthology.orchestra.run.smithi005.stdout: osd crush chooseleaf type = 0 2024-03-18T15:32:46.234 INFO:teuthology.orchestra.run.smithi005.stdout: auth debug = true 2024-03-18T15:32:46.234 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:32:46.234 INFO:teuthology.orchestra.run.smithi005.stdout: ms die on old message = true 2024-03-18T15:32:46.234 INFO:teuthology.orchestra.run.smithi005.stdout: ms die on bug = true 2024-03-18T15:32:46.234 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:32:46.234 INFO:teuthology.orchestra.run.smithi005.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-18T15:32:46.234 INFO:teuthology.orchestra.run.smithi005.stdout: mon pg warn max object skew = 0 2024-03-18T15:32:46.234 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:32:46.234 INFO:teuthology.orchestra.run.smithi005.stdout: # disable pg_autoscaler by default for new pools 2024-03-18T15:32:46.234 INFO:teuthology.orchestra.run.smithi005.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-18T15:32:46.234 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:32:46.234 INFO:teuthology.orchestra.run.smithi005.stdout: osd pool default size = 2 2024-03-18T15:32:46.234 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:32:46.234 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd allow primary affinity = true 2024-03-18T15:32:46.235 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd allow pg remap = true 2024-03-18T15:32:46.235 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on legacy crush tunables = false 2024-03-18T15:32:46.235 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on crush straw calc version zero = false 2024-03-18T15:32:46.235 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on no sortbitwise = false 2024-03-18T15:32:46.235 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on osd down out interval zero = false 2024-03-18T15:32:46.235 INFO:teuthology.orchestra.run.smithi005.stdout: mon warn on too few osds = false 2024-03-18T15:32:46.235 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-18T15:32:46.235 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-18T15:32:46.235 INFO:teuthology.orchestra.run.smithi005.stdout: mon_allow_pool_size_one = true 2024-03-18T15:32:46.235 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:32:46.235 INFO:teuthology.orchestra.run.smithi005.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-18T15:32:46.235 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:32:46.235 INFO:teuthology.orchestra.run.smithi005.stdout: osd default data pool replay window = 5 2024-03-18T15:32:46.235 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:32:46.236 INFO:teuthology.orchestra.run.smithi005.stdout: mon allow pool delete = true 2024-03-18T15:32:46.236 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:32:46.236 INFO:teuthology.orchestra.run.smithi005.stdout: mon cluster log file level = debug 2024-03-18T15:32:46.236 INFO:teuthology.orchestra.run.smithi005.stdout: debug asserts on shutdown = true 2024-03-18T15:32:46.236 INFO:teuthology.orchestra.run.smithi005.stdout: mon health detail to clog = false 2024-03-18T15:32:46.236 INFO:teuthology.orchestra.run.smithi005.stdout: mon host = "172.21.15.5,172.21.15.80,[v2:172.21.15.80:3301,v1:172.21.15.80:6790]" 2024-03-18T15:32:46.236 INFO:teuthology.orchestra.run.smithi005.stdout: fsid = ef54fe03-5aa7-4eb2-9b1e-66c9cc345b94 2024-03-18T15:32:46.236 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:32:46.236 INFO:teuthology.orchestra.run.smithi005.stdout:[osd] 2024-03-18T15:32:46.236 INFO:teuthology.orchestra.run.smithi005.stdout: osd journal size = 100 2024-03-18T15:32:46.236 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:32:46.236 INFO:teuthology.orchestra.run.smithi005.stdout: osd scrub load threshold = 5.0 2024-03-18T15:32:46.236 INFO:teuthology.orchestra.run.smithi005.stdout: osd scrub max interval = 600 2024-03-18T15:32:46.237 INFO:teuthology.orchestra.run.smithi005.stdout: osd mclock profile = high_recovery_ops 2024-03-18T15:32:46.237 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:32:46.237 INFO:teuthology.orchestra.run.smithi005.stdout: osd recover clone overlap = true 2024-03-18T15:32:46.237 INFO:teuthology.orchestra.run.smithi005.stdout: osd recovery max chunk = 1048576 2024-03-18T15:32:46.237 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:32:46.237 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug shutdown = true 2024-03-18T15:32:46.237 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug op order = true 2024-03-18T15:32:46.237 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug verify stray on activate = true 2024-03-18T15:32:46.237 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug trim objects = true 2024-03-18T15:32:46.237 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:32:46.237 INFO:teuthology.orchestra.run.smithi005.stdout: osd open classes on start = true 2024-03-18T15:32:46.237 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug pg log writeout = true 2024-03-18T15:32:46.237 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:32:46.237 INFO:teuthology.orchestra.run.smithi005.stdout: osd deep scrub update digest min age = 30 2024-03-18T15:32:46.237 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:32:46.238 INFO:teuthology.orchestra.run.smithi005.stdout: osd map max advance = 10 2024-03-18T15:32:46.238 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:32:46.238 INFO:teuthology.orchestra.run.smithi005.stdout: journal zero on create = true 2024-03-18T15:32:46.238 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:32:46.238 INFO:teuthology.orchestra.run.smithi005.stdout: filestore ondisk finisher threads = 3 2024-03-18T15:32:46.238 INFO:teuthology.orchestra.run.smithi005.stdout: filestore apply finisher threads = 3 2024-03-18T15:32:46.238 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:32:46.238 INFO:teuthology.orchestra.run.smithi005.stdout: bdev debug aio = true 2024-03-18T15:32:46.238 INFO:teuthology.orchestra.run.smithi005.stdout: osd debug misdirected ops = true 2024-03-18T15:32:46.238 INFO:teuthology.orchestra.run.smithi005.stdout: bdev async discard = True 2024-03-18T15:32:46.238 INFO:teuthology.orchestra.run.smithi005.stdout: bdev enable discard = True 2024-03-18T15:32:46.238 INFO:teuthology.orchestra.run.smithi005.stdout: bluestore allocator = bitmap 2024-03-18T15:32:46.238 INFO:teuthology.orchestra.run.smithi005.stdout: bluestore block size = 96636764160 2024-03-18T15:32:46.238 INFO:teuthology.orchestra.run.smithi005.stdout: bluestore fsck on mount = True 2024-03-18T15:32:46.238 INFO:teuthology.orchestra.run.smithi005.stdout: debug bluefs = 1/20 2024-03-18T15:32:46.239 INFO:teuthology.orchestra.run.smithi005.stdout: debug bluestore = 1/20 2024-03-18T15:32:46.239 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-03-18T15:32:46.239 INFO:teuthology.orchestra.run.smithi005.stdout: debug osd = 20 2024-03-18T15:32:46.239 INFO:teuthology.orchestra.run.smithi005.stdout: debug rocksdb = 4/10 2024-03-18T15:32:46.239 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd backfillfull_ratio = 0.85 2024-03-18T15:32:46.239 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd full ratio = 0.9 2024-03-18T15:32:46.239 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd nearfull ratio = 0.8 2024-03-18T15:32:46.239 INFO:teuthology.orchestra.run.smithi005.stdout: osd failsafe full ratio = 0.95 2024-03-18T15:32:46.239 INFO:teuthology.orchestra.run.smithi005.stdout: osd objectstore = bluestore 2024-03-18T15:32:46.239 INFO:teuthology.orchestra.run.smithi005.stdout: osd op complaint time = 180 2024-03-18T15:32:46.239 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:32:46.239 INFO:teuthology.orchestra.run.smithi005.stdout:[mgr] 2024-03-18T15:32:46.239 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-03-18T15:32:46.239 INFO:teuthology.orchestra.run.smithi005.stdout: debug mgr = 20 2024-03-18T15:32:46.239 INFO:teuthology.orchestra.run.smithi005.stdout: debug mon = 20 2024-03-18T15:32:46.240 INFO:teuthology.orchestra.run.smithi005.stdout: debug auth = 20 2024-03-18T15:32:46.240 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min pgs per osd = 4 2024-03-18T15:32:46.240 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min bytes per osd = 10 2024-03-18T15:32:46.240 INFO:teuthology.orchestra.run.smithi005.stdout: mgr/telemetry/nag = false 2024-03-18T15:32:46.240 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:32:46.240 INFO:teuthology.orchestra.run.smithi005.stdout:[mon] 2024-03-18T15:32:46.240 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-03-18T15:32:46.240 INFO:teuthology.orchestra.run.smithi005.stdout: debug mon = 20 2024-03-18T15:32:46.240 INFO:teuthology.orchestra.run.smithi005.stdout: debug paxos = 20 2024-03-18T15:32:46.240 INFO:teuthology.orchestra.run.smithi005.stdout: debug auth = 20 2024-03-18T15:32:46.240 INFO:teuthology.orchestra.run.smithi005.stdout: mon data avail warn = 5 2024-03-18T15:32:46.240 INFO:teuthology.orchestra.run.smithi005.stdout: mon mgr mkfs grace = 240 2024-03-18T15:32:46.240 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min pgs per osd = 4 2024-03-18T15:32:46.240 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd reporter subtree level = osd 2024-03-18T15:32:46.240 INFO:teuthology.orchestra.run.smithi005.stdout: mon osd prime pg temp = true 2024-03-18T15:32:46.240 INFO:teuthology.orchestra.run.smithi005.stdout: mon reweight min bytes per osd = 10 2024-03-18T15:32:46.241 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:32:46.241 INFO:teuthology.orchestra.run.smithi005.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-18T15:32:46.241 INFO:teuthology.orchestra.run.smithi005.stdout: auth mon ticket ttl = 660 # 11m 2024-03-18T15:32:46.241 INFO:teuthology.orchestra.run.smithi005.stdout: auth service ticket ttl = 240 # 4m 2024-03-18T15:32:46.241 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:32:46.241 INFO:teuthology.orchestra.run.smithi005.stdout: # don't complain about insecure global_id in the test suite 2024-03-18T15:32:46.241 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-18T15:32:46.241 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-18T15:32:46.241 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:32:46.241 INFO:teuthology.orchestra.run.smithi005.stdout: # 1m isn't quite enough 2024-03-18T15:32:46.241 INFO:teuthology.orchestra.run.smithi005.stdout: mon_down_mkfs_grace = 2m 2024-03-18T15:32:46.241 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:32:46.241 INFO:teuthology.orchestra.run.smithi005.stdout: mon_warn_on_filestore_osds = false 2024-03-18T15:32:46.242 INFO:teuthology.orchestra.run.smithi005.stdout: mon op complaint time = 120 2024-03-18T15:32:46.242 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:32:46.242 INFO:teuthology.orchestra.run.smithi005.stdout:[client] 2024-03-18T15:32:46.242 INFO:teuthology.orchestra.run.smithi005.stdout: rgw cache enabled = true 2024-03-18T15:32:46.242 INFO:teuthology.orchestra.run.smithi005.stdout: rgw enable ops log = true 2024-03-18T15:32:46.242 INFO:teuthology.orchestra.run.smithi005.stdout: rgw enable usage log = true 2024-03-18T15:32:46.242 INFO:teuthology.orchestra.run.smithi005.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-18T15:32:46.242 INFO:teuthology.orchestra.run.smithi005.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-18T15:32:46.242 INFO:teuthology.orchestra.run.smithi005.stdout: client mount timeout = 600 2024-03-18T15:32:46.242 INFO:teuthology.orchestra.run.smithi005.stdout: debug client = 20 2024-03-18T15:32:46.243 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-03-18T15:32:46.243 INFO:teuthology.orchestra.run.smithi005.stdout: pid file = "" 2024-03-18T15:32:46.243 INFO:teuthology.orchestra.run.smithi005.stdout: rados mon op timeout = 900 2024-03-18T15:32:46.243 INFO:teuthology.orchestra.run.smithi005.stdout: rados osd op timeout = 900 2024-03-18T15:32:46.243 INFO:teuthology.orchestra.run.smithi005.stdout:[mon.a] 2024-03-18T15:32:46.243 INFO:teuthology.orchestra.run.smithi005.stdout:[mon.b] 2024-03-18T15:32:46.243 INFO:teuthology.orchestra.run.smithi005.stdout:[mon.c] 2024-03-18T15:32:46.243 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.a] 2024-03-18T15:32:46.243 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.b] 2024-03-18T15:32:46.243 INFO:teuthology.orchestra.run.smithi005.stdout:[mds] 2024-03-18T15:32:46.243 INFO:teuthology.orchestra.run.smithi005.stdout: debug mds = 20 2024-03-18T15:32:46.243 INFO:teuthology.orchestra.run.smithi005.stdout: debug mds balancer = 20 2024-03-18T15:32:46.243 INFO:teuthology.orchestra.run.smithi005.stdout: debug ms = 1 2024-03-18T15:32:46.243 INFO:teuthology.orchestra.run.smithi005.stdout: mds debug frag = True 2024-03-18T15:32:46.244 INFO:teuthology.orchestra.run.smithi005.stdout: mds debug scatterstat = True 2024-03-18T15:32:46.244 INFO:teuthology.orchestra.run.smithi005.stdout: mds op complaint time = 180 2024-03-18T15:32:46.244 INFO:teuthology.orchestra.run.smithi005.stdout: mds verify scatter = True 2024-03-18T15:32:46.244 INFO:teuthology.orchestra.run.smithi005.stdout: osd op complaint time = 180 2024-03-18T15:32:46.244 INFO:teuthology.orchestra.run.smithi005.stdout: rados mon op timeout = 900 2024-03-18T15:32:46.244 INFO:teuthology.orchestra.run.smithi005.stdout: rados osd op timeout = 900 2024-03-18T15:32:46.245 INFO:teuthology.orchestra.run.smithi080.stdout:[global] 2024-03-18T15:32:46.245 INFO:teuthology.orchestra.run.smithi080.stdout: chdir = "" 2024-03-18T15:32:46.246 INFO:teuthology.orchestra.run.smithi080.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-18T15:32:46.246 INFO:teuthology.orchestra.run.smithi080.stdout: auth supported = cephx 2024-03-18T15:32:46.246 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-03-18T15:32:46.246 INFO:teuthology.orchestra.run.smithi080.stdout: filestore xattr use omap = true 2024-03-18T15:32:46.246 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-03-18T15:32:46.246 INFO:teuthology.orchestra.run.smithi080.stdout: mon clock drift allowed = 1.000 2024-03-18T15:32:46.246 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-03-18T15:32:46.246 INFO:teuthology.orchestra.run.smithi080.stdout: osd crush chooseleaf type = 0 2024-03-18T15:32:46.246 INFO:teuthology.orchestra.run.smithi080.stdout: auth debug = true 2024-03-18T15:32:46.247 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-03-18T15:32:46.247 INFO:teuthology.orchestra.run.smithi080.stdout: ms die on old message = true 2024-03-18T15:32:46.247 INFO:teuthology.orchestra.run.smithi080.stdout: ms die on bug = true 2024-03-18T15:32:46.247 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-03-18T15:32:46.247 INFO:teuthology.orchestra.run.smithi080.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-18T15:32:46.247 INFO:teuthology.orchestra.run.smithi080.stdout: mon pg warn max object skew = 0 2024-03-18T15:32:46.247 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-03-18T15:32:46.247 INFO:teuthology.orchestra.run.smithi080.stdout: # disable pg_autoscaler by default for new pools 2024-03-18T15:32:46.247 INFO:teuthology.orchestra.run.smithi080.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-18T15:32:46.247 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-03-18T15:32:46.247 INFO:teuthology.orchestra.run.smithi080.stdout: osd pool default size = 2 2024-03-18T15:32:46.247 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-03-18T15:32:46.247 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd allow primary affinity = true 2024-03-18T15:32:46.247 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd allow pg remap = true 2024-03-18T15:32:46.247 INFO:teuthology.orchestra.run.smithi080.stdout: mon warn on legacy crush tunables = false 2024-03-18T15:32:46.248 INFO:teuthology.orchestra.run.smithi080.stdout: mon warn on crush straw calc version zero = false 2024-03-18T15:32:46.248 INFO:teuthology.orchestra.run.smithi080.stdout: mon warn on no sortbitwise = false 2024-03-18T15:32:46.248 INFO:teuthology.orchestra.run.smithi080.stdout: mon warn on osd down out interval zero = false 2024-03-18T15:32:46.248 INFO:teuthology.orchestra.run.smithi080.stdout: mon warn on too few osds = false 2024-03-18T15:32:46.248 INFO:teuthology.orchestra.run.smithi080.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-18T15:32:46.248 INFO:teuthology.orchestra.run.smithi080.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-18T15:32:46.248 INFO:teuthology.orchestra.run.smithi080.stdout: mon_allow_pool_size_one = true 2024-03-18T15:32:46.248 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-03-18T15:32:46.248 INFO:teuthology.orchestra.run.smithi080.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-18T15:32:46.248 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-03-18T15:32:46.248 INFO:teuthology.orchestra.run.smithi080.stdout: osd default data pool replay window = 5 2024-03-18T15:32:46.248 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-03-18T15:32:46.248 INFO:teuthology.orchestra.run.smithi080.stdout: mon allow pool delete = true 2024-03-18T15:32:46.248 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-03-18T15:32:46.248 INFO:teuthology.orchestra.run.smithi080.stdout: mon cluster log file level = debug 2024-03-18T15:32:46.248 INFO:teuthology.orchestra.run.smithi080.stdout: debug asserts on shutdown = true 2024-03-18T15:32:46.249 INFO:teuthology.orchestra.run.smithi080.stdout: mon health detail to clog = false 2024-03-18T15:32:46.249 INFO:teuthology.orchestra.run.smithi080.stdout: mon host = "172.21.15.5,172.21.15.80,[v2:172.21.15.80:3301,v1:172.21.15.80:6790]" 2024-03-18T15:32:46.249 INFO:teuthology.orchestra.run.smithi080.stdout: fsid = ef54fe03-5aa7-4eb2-9b1e-66c9cc345b94 2024-03-18T15:32:46.249 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-03-18T15:32:46.249 INFO:teuthology.orchestra.run.smithi080.stdout:[osd] 2024-03-18T15:32:46.249 INFO:teuthology.orchestra.run.smithi080.stdout: osd journal size = 100 2024-03-18T15:32:46.249 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-03-18T15:32:46.249 INFO:teuthology.orchestra.run.smithi080.stdout: osd scrub load threshold = 5.0 2024-03-18T15:32:46.249 INFO:teuthology.orchestra.run.smithi080.stdout: osd scrub max interval = 600 2024-03-18T15:32:46.249 INFO:teuthology.orchestra.run.smithi080.stdout: osd mclock profile = high_recovery_ops 2024-03-18T15:32:46.249 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-03-18T15:32:46.249 INFO:teuthology.orchestra.run.smithi080.stdout: osd recover clone overlap = true 2024-03-18T15:32:46.249 INFO:teuthology.orchestra.run.smithi080.stdout: osd recovery max chunk = 1048576 2024-03-18T15:32:46.249 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-03-18T15:32:46.249 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug shutdown = true 2024-03-18T15:32:46.250 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug op order = true 2024-03-18T15:32:46.250 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug verify stray on activate = true 2024-03-18T15:32:46.250 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug trim objects = true 2024-03-18T15:32:46.250 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-03-18T15:32:46.250 INFO:teuthology.orchestra.run.smithi080.stdout: osd open classes on start = true 2024-03-18T15:32:46.250 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug pg log writeout = true 2024-03-18T15:32:46.250 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-03-18T15:32:46.250 INFO:teuthology.orchestra.run.smithi080.stdout: osd deep scrub update digest min age = 30 2024-03-18T15:32:46.250 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-03-18T15:32:46.250 INFO:teuthology.orchestra.run.smithi080.stdout: osd map max advance = 10 2024-03-18T15:32:46.250 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-03-18T15:32:46.250 INFO:teuthology.orchestra.run.smithi080.stdout: journal zero on create = true 2024-03-18T15:32:46.250 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-03-18T15:32:46.250 INFO:teuthology.orchestra.run.smithi080.stdout: filestore ondisk finisher threads = 3 2024-03-18T15:32:46.251 INFO:teuthology.orchestra.run.smithi080.stdout: filestore apply finisher threads = 3 2024-03-18T15:32:46.251 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-03-18T15:32:46.251 INFO:teuthology.orchestra.run.smithi080.stdout: bdev debug aio = true 2024-03-18T15:32:46.251 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug misdirected ops = true 2024-03-18T15:32:46.251 INFO:teuthology.orchestra.run.smithi080.stdout: bdev async discard = True 2024-03-18T15:32:46.251 INFO:teuthology.orchestra.run.smithi080.stdout: bdev enable discard = True 2024-03-18T15:32:46.251 INFO:teuthology.orchestra.run.smithi080.stdout: bluestore allocator = bitmap 2024-03-18T15:32:46.251 INFO:teuthology.orchestra.run.smithi080.stdout: bluestore block size = 96636764160 2024-03-18T15:32:46.251 INFO:teuthology.orchestra.run.smithi080.stdout: bluestore fsck on mount = True 2024-03-18T15:32:46.251 INFO:teuthology.orchestra.run.smithi080.stdout: debug bluefs = 1/20 2024-03-18T15:32:46.251 INFO:teuthology.orchestra.run.smithi080.stdout: debug bluestore = 1/20 2024-03-18T15:32:46.251 INFO:teuthology.orchestra.run.smithi080.stdout: debug ms = 1 2024-03-18T15:32:46.251 INFO:teuthology.orchestra.run.smithi080.stdout: debug osd = 20 2024-03-18T15:32:46.251 INFO:teuthology.orchestra.run.smithi080.stdout: debug rocksdb = 4/10 2024-03-18T15:32:46.252 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd backfillfull_ratio = 0.85 2024-03-18T15:32:46.252 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd full ratio = 0.9 2024-03-18T15:32:46.252 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd nearfull ratio = 0.8 2024-03-18T15:32:46.252 INFO:teuthology.orchestra.run.smithi080.stdout: osd failsafe full ratio = 0.95 2024-03-18T15:32:46.252 INFO:teuthology.orchestra.run.smithi080.stdout: osd objectstore = bluestore 2024-03-18T15:32:46.252 INFO:teuthology.orchestra.run.smithi080.stdout: osd op complaint time = 180 2024-03-18T15:32:46.252 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-03-18T15:32:46.252 INFO:teuthology.orchestra.run.smithi080.stdout:[mgr] 2024-03-18T15:32:46.252 INFO:teuthology.orchestra.run.smithi080.stdout: debug ms = 1 2024-03-18T15:32:46.252 INFO:teuthology.orchestra.run.smithi080.stdout: debug mgr = 20 2024-03-18T15:32:46.252 INFO:teuthology.orchestra.run.smithi080.stdout: debug mon = 20 2024-03-18T15:32:46.252 INFO:teuthology.orchestra.run.smithi080.stdout: debug auth = 20 2024-03-18T15:32:46.252 INFO:teuthology.orchestra.run.smithi080.stdout: mon reweight min pgs per osd = 4 2024-03-18T15:32:46.252 INFO:teuthology.orchestra.run.smithi080.stdout: mon reweight min bytes per osd = 10 2024-03-18T15:32:46.252 INFO:teuthology.orchestra.run.smithi080.stdout: mgr/telemetry/nag = false 2024-03-18T15:32:46.253 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-03-18T15:32:46.253 INFO:teuthology.orchestra.run.smithi080.stdout:[mon] 2024-03-18T15:32:46.253 INFO:teuthology.orchestra.run.smithi080.stdout: debug ms = 1 2024-03-18T15:32:46.253 INFO:teuthology.orchestra.run.smithi080.stdout: debug mon = 20 2024-03-18T15:32:46.253 INFO:teuthology.orchestra.run.smithi080.stdout: debug paxos = 20 2024-03-18T15:32:46.253 INFO:teuthology.orchestra.run.smithi080.stdout: debug auth = 20 2024-03-18T15:32:46.253 INFO:teuthology.orchestra.run.smithi080.stdout: mon data avail warn = 5 2024-03-18T15:32:46.253 INFO:teuthology.orchestra.run.smithi080.stdout: mon mgr mkfs grace = 240 2024-03-18T15:32:46.253 INFO:teuthology.orchestra.run.smithi080.stdout: mon reweight min pgs per osd = 4 2024-03-18T15:32:46.253 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd reporter subtree level = osd 2024-03-18T15:32:46.253 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd prime pg temp = true 2024-03-18T15:32:46.253 INFO:teuthology.orchestra.run.smithi080.stdout: mon reweight min bytes per osd = 10 2024-03-18T15:32:46.253 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-03-18T15:32:46.253 INFO:teuthology.orchestra.run.smithi080.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-18T15:32:46.253 INFO:teuthology.orchestra.run.smithi080.stdout: auth mon ticket ttl = 660 # 11m 2024-03-18T15:32:46.253 INFO:teuthology.orchestra.run.smithi080.stdout: auth service ticket ttl = 240 # 4m 2024-03-18T15:32:46.254 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-03-18T15:32:46.254 INFO:teuthology.orchestra.run.smithi080.stdout: # don't complain about insecure global_id in the test suite 2024-03-18T15:32:46.254 INFO:teuthology.orchestra.run.smithi080.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-18T15:32:46.254 INFO:teuthology.orchestra.run.smithi080.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-18T15:32:46.254 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-03-18T15:32:46.254 INFO:teuthology.orchestra.run.smithi080.stdout: # 1m isn't quite enough 2024-03-18T15:32:46.254 INFO:teuthology.orchestra.run.smithi080.stdout: mon_down_mkfs_grace = 2m 2024-03-18T15:32:46.254 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-03-18T15:32:46.254 INFO:teuthology.orchestra.run.smithi080.stdout: mon_warn_on_filestore_osds = false 2024-03-18T15:32:46.254 INFO:teuthology.orchestra.run.smithi080.stdout: mon op complaint time = 120 2024-03-18T15:32:46.254 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-03-18T15:32:46.254 INFO:teuthology.orchestra.run.smithi080.stdout:[client] 2024-03-18T15:32:46.254 INFO:teuthology.orchestra.run.smithi080.stdout: rgw cache enabled = true 2024-03-18T15:32:46.255 INFO:teuthology.orchestra.run.smithi080.stdout: rgw enable ops log = true 2024-03-18T15:32:46.255 INFO:teuthology.orchestra.run.smithi080.stdout: rgw enable usage log = true 2024-03-18T15:32:46.255 INFO:teuthology.orchestra.run.smithi080.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-18T15:32:46.255 INFO:teuthology.orchestra.run.smithi080.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-18T15:32:46.255 INFO:teuthology.orchestra.run.smithi080.stdout: client mount timeout = 600 2024-03-18T15:32:46.255 INFO:teuthology.orchestra.run.smithi080.stdout: debug client = 20 2024-03-18T15:32:46.255 INFO:teuthology.orchestra.run.smithi080.stdout: debug ms = 1 2024-03-18T15:32:46.255 INFO:teuthology.orchestra.run.smithi080.stdout: pid file = "" 2024-03-18T15:32:46.255 INFO:teuthology.orchestra.run.smithi080.stdout: rados mon op timeout = 900 2024-03-18T15:32:46.255 INFO:teuthology.orchestra.run.smithi080.stdout: rados osd op timeout = 900 2024-03-18T15:32:46.255 INFO:teuthology.orchestra.run.smithi080.stdout:[mon.a] 2024-03-18T15:32:46.255 INFO:teuthology.orchestra.run.smithi080.stdout:[mon.b] 2024-03-18T15:32:46.255 INFO:teuthology.orchestra.run.smithi080.stdout:[mon.c] 2024-03-18T15:32:46.255 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.a] 2024-03-18T15:32:46.255 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.b] 2024-03-18T15:32:46.255 INFO:teuthology.orchestra.run.smithi080.stdout:[mds] 2024-03-18T15:32:46.256 INFO:teuthology.orchestra.run.smithi080.stdout: debug mds = 20 2024-03-18T15:32:46.256 INFO:teuthology.orchestra.run.smithi080.stdout: debug mds balancer = 20 2024-03-18T15:32:46.256 INFO:teuthology.orchestra.run.smithi080.stdout: debug ms = 1 2024-03-18T15:32:46.256 INFO:teuthology.orchestra.run.smithi080.stdout: mds debug frag = True 2024-03-18T15:32:46.256 INFO:teuthology.orchestra.run.smithi080.stdout: mds debug scatterstat = True 2024-03-18T15:32:46.256 INFO:teuthology.orchestra.run.smithi080.stdout: mds op complaint time = 180 2024-03-18T15:32:46.256 INFO:teuthology.orchestra.run.smithi080.stdout: mds verify scatter = True 2024-03-18T15:32:46.256 INFO:teuthology.orchestra.run.smithi080.stdout: osd op complaint time = 180 2024-03-18T15:32:46.256 INFO:teuthology.orchestra.run.smithi080.stdout: rados mon op timeout = 900 2024-03-18T15:32:46.256 INFO:teuthology.orchestra.run.smithi080.stdout: rados osd op timeout = 900 2024-03-18T15:32:46.257 INFO:tasks.ceph:Creating admin key on mon.a... 2024-03-18T15:32:46.257 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-03-18T15:32:46.333 INFO:tasks.ceph:Copying monmap to all nodes... 2024-03-18T15:32:46.333 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-03-18T15:32:46.333 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-03-18T15:32:46.383 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-03-18T15:32:46.383 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-03-18T15:32:46.430 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi005.front.sepia.ceph.com 2024-03-18T15:32:46.430 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-03-18T15:32:46.430 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-18T15:32:46.431 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-18T15:32:46.498 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-03-18T15:32:46.498 DEBUG:teuthology.orchestra.run.smithi005:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-18T15:32:46.550 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi080.front.sepia.ceph.com 2024-03-18T15:32:46.551 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-03-18T15:32:46.551 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-18T15:32:46.551 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-18T15:32:46.574 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-03-18T15:32:46.575 DEBUG:teuthology.orchestra.run.smithi080:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-18T15:32:46.623 INFO:tasks.ceph:Setting up mon nodes... 2024-03-18T15:32:46.623 INFO:tasks.ceph:Setting up mgr nodes... 2024-03-18T15:32:46.624 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-03-18T15:32:46.666 INFO:teuthology.orchestra.run.smithi005.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-03-18T15:32:46.669 DEBUG:teuthology.orchestra.run.smithi080:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-03-18T15:32:46.711 INFO:teuthology.orchestra.run.smithi080.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-03-18T15:32:46.719 INFO:tasks.ceph:Setting up mds nodes... 2024-03-18T15:32:46.720 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-03-18T15:32:46.759 INFO:teuthology.orchestra.run.smithi005.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-03-18T15:32:46.762 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-03-18T15:32:46.817 DEBUG:teuthology.orchestra.run.smithi080:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-03-18T15:32:46.860 INFO:teuthology.orchestra.run.smithi080.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-03-18T15:32:46.863 DEBUG:teuthology.orchestra.run.smithi080:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-03-18T15:32:46.919 INFO:tasks.ceph_client:Setting up client nodes... 2024-03-18T15:32:46.920 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-18T15:32:46.955 INFO:teuthology.orchestra.run.smithi005.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-03-18T15:32:46.967 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-03-18T15:32:46.967 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi005.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi080.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-18T15:32:46.967 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-03-18T15:32:47.022 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-18T15:32:47.022 INFO:tasks.ceph:role: osd.0 2024-03-18T15:32:47.022 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi005.front.sepia.ceph.com 2024-03-18T15:32:47.022 DEBUG:teuthology.orchestra.run.smithi005:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-18T15:32:47.078 INFO:teuthology.orchestra.run.smithi005.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-18T15:32:47.079 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-18T15:32:47.079 INFO:teuthology.orchestra.run.smithi005.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-18T15:32:47.079 INFO:teuthology.orchestra.run.smithi005.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-18T15:32:47.079 INFO:teuthology.orchestra.run.smithi005.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-18T15:32:47.079 INFO:teuthology.orchestra.run.smithi005.stdout: = sunit=0 swidth=0 blks 2024-03-18T15:32:47.079 INFO:teuthology.orchestra.run.smithi005.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-18T15:32:47.079 INFO:teuthology.orchestra.run.smithi005.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-18T15:32:47.079 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-18T15:32:47.079 INFO:teuthology.orchestra.run.smithi005.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-18T15:32:47.104 INFO:teuthology.orchestra.run.smithi005.stdout:Discarding blocks...Done. 2024-03-18T15:32:47.106 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi005.front.sepia.ceph.com -o noatime 2024-03-18T15:32:47.106 DEBUG:teuthology.orchestra.run.smithi005:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-03-18T15:32:47.164 DEBUG:teuthology.orchestra.run.smithi005:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-03-18T15:32:47.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T15:32:47.215 INFO:teuthology.orchestra.run.smithi005.stderr:sudo: /sbin/restorecon: command not found 2024-03-18T15:32:47.215 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-03-18T15:32:47.269 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-18T15:32:47.270 INFO:tasks.ceph:role: osd.1 2024-03-18T15:32:47.270 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi005.front.sepia.ceph.com 2024-03-18T15:32:47.270 DEBUG:teuthology.orchestra.run.smithi005:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-18T15:32:47.327 INFO:teuthology.orchestra.run.smithi005.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-18T15:32:47.327 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-18T15:32:47.327 INFO:teuthology.orchestra.run.smithi005.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-18T15:32:47.328 INFO:teuthology.orchestra.run.smithi005.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-18T15:32:47.328 INFO:teuthology.orchestra.run.smithi005.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-18T15:32:47.328 INFO:teuthology.orchestra.run.smithi005.stdout: = sunit=0 swidth=0 blks 2024-03-18T15:32:47.328 INFO:teuthology.orchestra.run.smithi005.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-18T15:32:47.328 INFO:teuthology.orchestra.run.smithi005.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-18T15:32:47.328 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-18T15:32:47.328 INFO:teuthology.orchestra.run.smithi005.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-18T15:32:47.353 INFO:teuthology.orchestra.run.smithi005.stdout:Discarding blocks...Done. 2024-03-18T15:32:47.355 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi005.front.sepia.ceph.com -o noatime 2024-03-18T15:32:47.355 DEBUG:teuthology.orchestra.run.smithi005:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-03-18T15:32:47.415 DEBUG:teuthology.orchestra.run.smithi005:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-03-18T15:32:47.466 INFO:teuthology.orchestra.run.smithi005.stderr:sudo: /sbin/restorecon: command not found 2024-03-18T15:32:47.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T15:32:47.467 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-03-18T15:32:47.522 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-18T15:32:47.522 INFO:tasks.ceph:role: osd.2 2024-03-18T15:32:47.522 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi005.front.sepia.ceph.com 2024-03-18T15:32:47.522 DEBUG:teuthology.orchestra.run.smithi005:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-18T15:32:47.578 INFO:teuthology.orchestra.run.smithi005.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-18T15:32:47.578 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-18T15:32:47.578 INFO:teuthology.orchestra.run.smithi005.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-18T15:32:47.578 INFO:teuthology.orchestra.run.smithi005.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-18T15:32:47.578 INFO:teuthology.orchestra.run.smithi005.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-18T15:32:47.578 INFO:teuthology.orchestra.run.smithi005.stdout: = sunit=0 swidth=0 blks 2024-03-18T15:32:47.578 INFO:teuthology.orchestra.run.smithi005.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-18T15:32:47.578 INFO:teuthology.orchestra.run.smithi005.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-18T15:32:47.578 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-18T15:32:47.578 INFO:teuthology.orchestra.run.smithi005.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-18T15:32:47.604 INFO:teuthology.orchestra.run.smithi005.stdout:Discarding blocks...Done. 2024-03-18T15:32:47.606 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi005.front.sepia.ceph.com -o noatime 2024-03-18T15:32:47.606 DEBUG:teuthology.orchestra.run.smithi005:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-03-18T15:32:47.666 DEBUG:teuthology.orchestra.run.smithi005:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-03-18T15:32:47.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T15:32:47.723 INFO:teuthology.orchestra.run.smithi005.stderr:sudo: /sbin/restorecon: command not found 2024-03-18T15:32:47.724 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-03-18T15:32:47.777 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-18T15:32:47.777 INFO:tasks.ceph:role: osd.3 2024-03-18T15:32:47.778 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi005.front.sepia.ceph.com 2024-03-18T15:32:47.778 DEBUG:teuthology.orchestra.run.smithi005:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-18T15:32:47.831 INFO:teuthology.orchestra.run.smithi005.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-18T15:32:47.831 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-18T15:32:47.831 INFO:teuthology.orchestra.run.smithi005.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-18T15:32:47.831 INFO:teuthology.orchestra.run.smithi005.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-18T15:32:47.831 INFO:teuthology.orchestra.run.smithi005.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-18T15:32:47.831 INFO:teuthology.orchestra.run.smithi005.stdout: = sunit=0 swidth=0 blks 2024-03-18T15:32:47.831 INFO:teuthology.orchestra.run.smithi005.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-18T15:32:47.831 INFO:teuthology.orchestra.run.smithi005.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-18T15:32:47.831 INFO:teuthology.orchestra.run.smithi005.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-18T15:32:47.832 INFO:teuthology.orchestra.run.smithi005.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-18T15:32:47.857 INFO:teuthology.orchestra.run.smithi005.stdout:Discarding blocks...Done. 2024-03-18T15:32:47.858 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi005.front.sepia.ceph.com -o noatime 2024-03-18T15:32:47.859 DEBUG:teuthology.orchestra.run.smithi005:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-03-18T15:32:47.916 DEBUG:teuthology.orchestra.run.smithi005:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-03-18T15:32:47.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T15:32:47.968 INFO:teuthology.orchestra.run.smithi005.stderr:sudo: /sbin/restorecon: command not found 2024-03-18T15:32:47.969 DEBUG:teuthology.orchestra.run.smithi005:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-18T15:32:48.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:32:48.074+0000 7f8464f527c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-03-18T15:32:48.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:32:48.074+0000 7f8464f527c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-03-18T15:32:48.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:32:48.074+0000 7f8464f527c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-18T15:32:48.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:32:48.074+0000 7f8464f527c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-18T15:32:48.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:32:48.074+0000 7f8464f527c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-18T15:32:48.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:32:48.074+0000 7f8464f527c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-03-18T15:32:51.039 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-18T15:32:51.053 DEBUG:teuthology.orchestra.run.smithi005:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-18T15:32:51.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:32:51.134+0000 7f842fe3f7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-03-18T15:32:51.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:32:51.134+0000 7f842fe3f7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-03-18T15:32:51.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:32:51.134+0000 7f842fe3f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-18T15:32:51.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:32:51.134+0000 7f842fe3f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-18T15:32:51.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:32:51.134+0000 7f842fe3f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-18T15:32:51.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:32:51.138+0000 7f842fe3f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-03-18T15:32:54.089 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-18T15:32:54.143 DEBUG:teuthology.orchestra.run.smithi005:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-18T15:32:54.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:32:54.226+0000 7f99fad2f7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-03-18T15:32:54.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:32:54.226+0000 7f99fad2f7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-03-18T15:32:54.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:32:54.226+0000 7f99fad2f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-18T15:32:54.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:32:54.226+0000 7f99fad2f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-18T15:32:54.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:32:54.226+0000 7f99fad2f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-18T15:32:54.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:32:54.226+0000 7f99fad2f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-03-18T15:32:57.185 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-18T15:32:57.247 DEBUG:teuthology.orchestra.run.smithi005:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-18T15:32:57.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:32:57.330+0000 7fc17cba67c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-03-18T15:32:57.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:32:57.330+0000 7fc17cba67c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-03-18T15:32:57.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:32:57.330+0000 7fc17cba67c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-18T15:32:57.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:32:57.330+0000 7fc17cba67c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-18T15:32:57.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:32:57.330+0000 7fc17cba67c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-18T15:32:57.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:32:57.330+0000 7fc17cba67c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-03-18T15:33:00.297 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-18T15:33:00.354 DEBUG:teuthology.orchestra.run.smithi080:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-03-18T15:33:00.369 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-18T15:33:00.369 INFO:tasks.ceph:role: osd.4 2024-03-18T15:33:00.370 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi080.front.sepia.ceph.com 2024-03-18T15:33:00.370 DEBUG:teuthology.orchestra.run.smithi080:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-18T15:33:00.436 INFO:teuthology.orchestra.run.smithi080.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-18T15:33:00.436 INFO:teuthology.orchestra.run.smithi080.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-18T15:33:00.436 INFO:teuthology.orchestra.run.smithi080.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-18T15:33:00.436 INFO:teuthology.orchestra.run.smithi080.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-18T15:33:00.436 INFO:teuthology.orchestra.run.smithi080.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-18T15:33:00.436 INFO:teuthology.orchestra.run.smithi080.stdout: = sunit=0 swidth=0 blks 2024-03-18T15:33:00.436 INFO:teuthology.orchestra.run.smithi080.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-18T15:33:00.437 INFO:teuthology.orchestra.run.smithi080.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-18T15:33:00.437 INFO:teuthology.orchestra.run.smithi080.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-18T15:33:00.437 INFO:teuthology.orchestra.run.smithi080.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-18T15:33:00.887 INFO:teuthology.orchestra.run.smithi080.stdout:Discarding blocks...Done. 2024-03-18T15:33:00.889 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi080.front.sepia.ceph.com -o noatime 2024-03-18T15:33:00.889 DEBUG:teuthology.orchestra.run.smithi080:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-03-18T15:33:00.989 DEBUG:teuthology.orchestra.run.smithi080:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-03-18T15:33:00.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T15:33:00.999 INFO:teuthology.orchestra.run.smithi080.stderr:sudo: /sbin/restorecon: command not found 2024-03-18T15:33:00.999 DEBUG:teuthology.orchestra.run.smithi080:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-03-18T15:33:01.050 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-18T15:33:01.050 INFO:tasks.ceph:role: osd.5 2024-03-18T15:33:01.050 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi080.front.sepia.ceph.com 2024-03-18T15:33:01.050 DEBUG:teuthology.orchestra.run.smithi080:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-18T15:33:01.112 INFO:teuthology.orchestra.run.smithi080.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-18T15:33:01.113 INFO:teuthology.orchestra.run.smithi080.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-18T15:33:01.113 INFO:teuthology.orchestra.run.smithi080.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-18T15:33:01.113 INFO:teuthology.orchestra.run.smithi080.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-18T15:33:01.113 INFO:teuthology.orchestra.run.smithi080.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-18T15:33:01.113 INFO:teuthology.orchestra.run.smithi080.stdout: = sunit=0 swidth=0 blks 2024-03-18T15:33:01.113 INFO:teuthology.orchestra.run.smithi080.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-18T15:33:01.113 INFO:teuthology.orchestra.run.smithi080.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-18T15:33:01.113 INFO:teuthology.orchestra.run.smithi080.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-18T15:33:01.113 INFO:teuthology.orchestra.run.smithi080.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-18T15:33:01.571 INFO:teuthology.orchestra.run.smithi080.stdout:Discarding blocks...Done. 2024-03-18T15:33:01.572 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi080.front.sepia.ceph.com -o noatime 2024-03-18T15:33:01.573 DEBUG:teuthology.orchestra.run.smithi080:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-03-18T15:33:01.706 DEBUG:teuthology.orchestra.run.smithi080:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-03-18T15:33:01.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T15:33:01.722 INFO:teuthology.orchestra.run.smithi080.stderr:sudo: /sbin/restorecon: command not found 2024-03-18T15:33:01.723 DEBUG:teuthology.orchestra.run.smithi080:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-03-18T15:33:01.783 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-18T15:33:01.783 INFO:tasks.ceph:role: osd.6 2024-03-18T15:33:01.783 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi080.front.sepia.ceph.com 2024-03-18T15:33:01.784 DEBUG:teuthology.orchestra.run.smithi080:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-18T15:33:01.850 INFO:teuthology.orchestra.run.smithi080.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-18T15:33:01.851 INFO:teuthology.orchestra.run.smithi080.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-18T15:33:01.851 INFO:teuthology.orchestra.run.smithi080.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-18T15:33:01.851 INFO:teuthology.orchestra.run.smithi080.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-18T15:33:01.851 INFO:teuthology.orchestra.run.smithi080.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-18T15:33:01.851 INFO:teuthology.orchestra.run.smithi080.stdout: = sunit=0 swidth=0 blks 2024-03-18T15:33:01.851 INFO:teuthology.orchestra.run.smithi080.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-18T15:33:01.851 INFO:teuthology.orchestra.run.smithi080.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-18T15:33:01.851 INFO:teuthology.orchestra.run.smithi080.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-18T15:33:01.851 INFO:teuthology.orchestra.run.smithi080.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-18T15:33:02.330 INFO:teuthology.orchestra.run.smithi080.stdout:Discarding blocks...Done. 2024-03-18T15:33:02.332 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi080.front.sepia.ceph.com -o noatime 2024-03-18T15:33:02.333 DEBUG:teuthology.orchestra.run.smithi080:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-03-18T15:33:02.482 DEBUG:teuthology.orchestra.run.smithi080:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-03-18T15:33:02.493 INFO:teuthology.orchestra.run.smithi080.stderr:sudo: /sbin/restorecon: command not found 2024-03-18T15:33:02.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T15:33:02.494 DEBUG:teuthology.orchestra.run.smithi080:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-03-18T15:33:02.549 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-18T15:33:02.550 INFO:tasks.ceph:role: osd.7 2024-03-18T15:33:02.550 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi080.front.sepia.ceph.com 2024-03-18T15:33:02.550 DEBUG:teuthology.orchestra.run.smithi080:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-18T15:33:02.614 INFO:teuthology.orchestra.run.smithi080.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-18T15:33:02.614 INFO:teuthology.orchestra.run.smithi080.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-18T15:33:02.614 INFO:teuthology.orchestra.run.smithi080.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-18T15:33:02.614 INFO:teuthology.orchestra.run.smithi080.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-18T15:33:02.615 INFO:teuthology.orchestra.run.smithi080.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-18T15:33:02.615 INFO:teuthology.orchestra.run.smithi080.stdout: = sunit=0 swidth=0 blks 2024-03-18T15:33:02.615 INFO:teuthology.orchestra.run.smithi080.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-18T15:33:02.615 INFO:teuthology.orchestra.run.smithi080.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-18T15:33:02.615 INFO:teuthology.orchestra.run.smithi080.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-18T15:33:02.615 INFO:teuthology.orchestra.run.smithi080.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-18T15:33:03.039 INFO:teuthology.orchestra.run.smithi080.stdout:Discarding blocks...Done. 2024-03-18T15:33:03.041 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi080.front.sepia.ceph.com -o noatime 2024-03-18T15:33:03.041 DEBUG:teuthology.orchestra.run.smithi080:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-03-18T15:33:03.149 DEBUG:teuthology.orchestra.run.smithi080:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-03-18T15:33:03.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T15:33:03.160 INFO:teuthology.orchestra.run.smithi080.stderr:sudo: /sbin/restorecon: command not found 2024-03-18T15:33:03.161 DEBUG:teuthology.orchestra.run.smithi080:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-18T15:33:03.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:03.267+0000 7faaa4d097c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-03-18T15:33:03.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:03.267+0000 7faaa4d097c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-03-18T15:33:03.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:03.267+0000 7faaa4d097c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-18T15:33:03.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:03.267+0000 7faaa4d097c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-18T15:33:03.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:03.267+0000 7faaa4d097c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-18T15:33:03.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:03.271+0000 7faaa4d097c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-03-18T15:33:06.252 DEBUG:teuthology.orchestra.run.smithi080:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-18T15:33:06.265 DEBUG:teuthology.orchestra.run.smithi080:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-18T15:33:06.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:06.342+0000 7fae7e6347c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-03-18T15:33:06.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:06.342+0000 7fae7e6347c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-03-18T15:33:06.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:06.342+0000 7fae7e6347c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-18T15:33:06.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:06.342+0000 7fae7e6347c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-18T15:33:06.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:06.342+0000 7fae7e6347c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-18T15:33:06.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:06.346+0000 7fae7e6347c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-03-18T15:33:09.320 DEBUG:teuthology.orchestra.run.smithi080:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-18T15:33:09.377 DEBUG:teuthology.orchestra.run.smithi080:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-18T15:33:09.460 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:09.458+0000 7f0f340737c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-03-18T15:33:09.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:09.458+0000 7f0f340737c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-03-18T15:33:09.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:09.458+0000 7f0f340737c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-18T15:33:09.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:09.458+0000 7f0f340737c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-18T15:33:09.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:09.458+0000 7f0f340737c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-18T15:33:09.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:09.458+0000 7f0f340737c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-03-18T15:33:12.431 DEBUG:teuthology.orchestra.run.smithi080:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-03-18T15:33:12.486 DEBUG:teuthology.orchestra.run.smithi080:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-18T15:33:12.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:12.562+0000 7f60f1f017c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-03-18T15:33:12.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:12.562+0000 7f60f1f017c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-03-18T15:33:12.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:12.562+0000 7f60f1f017c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-18T15:33:12.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:12.562+0000 7f60f1f017c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-18T15:33:12.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:12.562+0000 7f60f1f017c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-18T15:33:12.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:12.562+0000 7f60f1f017c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-03-18T15:33:14.450 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-18T15:33:14.453 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-18T15:33:15.546 DEBUG:teuthology.orchestra.run.smithi080:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-03-18T15:33:15.559 INFO:tasks.ceph:Reading keys from all nodes... 2024-03-18T15:33:15.560 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-03-18T15:33:15.560 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-03-18T15:33:15.574 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-03-18T15:33:15.574 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-03-18T15:33:15.629 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-03-18T15:33:15.630 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-03-18T15:33:15.686 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-03-18T15:33:15.686 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-03-18T15:33:15.742 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-03-18T15:33:15.742 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-03-18T15:33:15.797 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-03-18T15:33:15.798 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-03-18T15:33:15.853 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-03-18T15:33:15.853 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-03-18T15:33:15.866 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-03-18T15:33:15.866 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-03-18T15:33:15.920 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-03-18T15:33:15.920 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-03-18T15:33:15.972 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-03-18T15:33:15.972 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-03-18T15:33:16.024 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-03-18T15:33:16.025 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-03-18T15:33:16.080 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-03-18T15:33:16.080 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-03-18T15:33:16.133 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-03-18T15:33:16.133 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-03-18T15:33:16.140 INFO:tasks.ceph:Adding keys to all mons... 2024-03-18T15:33:16.140 DEBUG:teuthology.orchestra.run.smithi005:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-18T15:33:16.183 DEBUG:teuthology.orchestra.run.smithi080:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-18T15:33:16.193 INFO:teuthology.orchestra.run.smithi005.stdout:[mgr.y] 2024-03-18T15:33:16.193 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQCeXvhlZ1/DJxAAhLPs51ARBSnFH+qu0h+1Jw== 2024-03-18T15:33:16.193 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.a] 2024-03-18T15:33:16.193 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQCeXvhllrlWLRAAeoM9rVm/NIT0Z1HQC7hlrg== 2024-03-18T15:33:16.193 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.0] 2024-03-18T15:33:16.193 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQCgXvhlJ3S2BBAAX7b5T6UKtrnOhSIhy2V5nw== 2024-03-18T15:33:16.194 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.1] 2024-03-18T15:33:16.194 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQCjXvhl/y82CBAAJXuiwwphz1dLCegtoBw/sQ== 2024-03-18T15:33:16.194 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.2] 2024-03-18T15:33:16.194 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQCmXvhlU+yBDRAA9DtMz35eiXflG1zQuqEScg== 2024-03-18T15:33:16.194 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.3] 2024-03-18T15:33:16.194 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQCpXvhlsyq3ExAAY53wP1De3pPBaZuKF7kFrw== 2024-03-18T15:33:16.194 INFO:teuthology.orchestra.run.smithi005.stdout:[mgr.x] 2024-03-18T15:33:16.194 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQCeXvhlF4OfKhAAost72x1wdCRqzhgIdqEw1w== 2024-03-18T15:33:16.194 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.b] 2024-03-18T15:33:16.195 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQCeXvhlp5F9MxAAParGkBY7Sc9gOUKZVLxBzg== 2024-03-18T15:33:16.195 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.4] 2024-03-18T15:33:16.195 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQCvXvhlwwEnEBAAS9wCX93LDdiGtiyKWy/1lA== 2024-03-18T15:33:16.195 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.5] 2024-03-18T15:33:16.195 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQCyXvhlxC7HFBAACkH4xW+XsH+LMc384ulWow== 2024-03-18T15:33:16.195 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.6] 2024-03-18T15:33:16.195 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQC1XvhlgHeLGxAA2Q5gGrjSpaVmIdwunlRBBw== 2024-03-18T15:33:16.195 INFO:teuthology.orchestra.run.smithi005.stdout:[osd.7] 2024-03-18T15:33:16.195 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQC4XvhlRB7XIRAAqxxe4tWUrF3n5VK/O4PHcg== 2024-03-18T15:33:16.195 INFO:teuthology.orchestra.run.smithi005.stdout:[client.0] 2024-03-18T15:33:16.195 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQCeXvhlDBsBORAA9deuFWz05eGfJp1vj5Vlkg== 2024-03-18T15:33:16.197 INFO:teuthology.orchestra.run.smithi080.stdout:[mgr.y] 2024-03-18T15:33:16.197 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQCeXvhlZ1/DJxAAhLPs51ARBSnFH+qu0h+1Jw== 2024-03-18T15:33:16.197 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.a] 2024-03-18T15:33:16.197 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQCeXvhllrlWLRAAeoM9rVm/NIT0Z1HQC7hlrg== 2024-03-18T15:33:16.197 INFO:teuthology.orchestra.run.smithi080.stdout:[osd.0] 2024-03-18T15:33:16.197 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQCgXvhlJ3S2BBAAX7b5T6UKtrnOhSIhy2V5nw== 2024-03-18T15:33:16.197 INFO:teuthology.orchestra.run.smithi080.stdout:[osd.1] 2024-03-18T15:33:16.197 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQCjXvhl/y82CBAAJXuiwwphz1dLCegtoBw/sQ== 2024-03-18T15:33:16.197 INFO:teuthology.orchestra.run.smithi080.stdout:[osd.2] 2024-03-18T15:33:16.197 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQCmXvhlU+yBDRAA9DtMz35eiXflG1zQuqEScg== 2024-03-18T15:33:16.197 INFO:teuthology.orchestra.run.smithi080.stdout:[osd.3] 2024-03-18T15:33:16.198 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQCpXvhlsyq3ExAAY53wP1De3pPBaZuKF7kFrw== 2024-03-18T15:33:16.198 INFO:teuthology.orchestra.run.smithi080.stdout:[mgr.x] 2024-03-18T15:33:16.198 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQCeXvhlF4OfKhAAost72x1wdCRqzhgIdqEw1w== 2024-03-18T15:33:16.198 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.b] 2024-03-18T15:33:16.198 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQCeXvhlp5F9MxAAParGkBY7Sc9gOUKZVLxBzg== 2024-03-18T15:33:16.198 INFO:teuthology.orchestra.run.smithi080.stdout:[osd.4] 2024-03-18T15:33:16.198 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQCvXvhlwwEnEBAAS9wCX93LDdiGtiyKWy/1lA== 2024-03-18T15:33:16.198 INFO:teuthology.orchestra.run.smithi080.stdout:[osd.5] 2024-03-18T15:33:16.198 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQCyXvhlxC7HFBAACkH4xW+XsH+LMc384ulWow== 2024-03-18T15:33:16.198 INFO:teuthology.orchestra.run.smithi080.stdout:[osd.6] 2024-03-18T15:33:16.198 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQC1XvhlgHeLGxAA2Q5gGrjSpaVmIdwunlRBBw== 2024-03-18T15:33:16.198 INFO:teuthology.orchestra.run.smithi080.stdout:[osd.7] 2024-03-18T15:33:16.198 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQC4XvhlRB7XIRAAqxxe4tWUrF3n5VK/O4PHcg== 2024-03-18T15:33:16.199 INFO:teuthology.orchestra.run.smithi080.stdout:[client.0] 2024-03-18T15:33:16.199 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQCeXvhlDBsBORAA9deuFWz05eGfJp1vj5Vlkg== 2024-03-18T15:33:16.199 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-18T15:33:16.239 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-18T15:33:16.285 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-18T15:33:16.288 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-18T15:33:16.333 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-18T15:33:16.336 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-18T15:33:16.417 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-18T15:33:16.423 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-18T15:33:16.507 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-18T15:33:16.511 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-18T15:33:16.592 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-18T15:33:16.594 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-18T15:33:16.676 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-18T15:33:16.679 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-18T15:33:16.761 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-18T15:33:16.763 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-18T15:33:16.848 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-18T15:33:16.851 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-18T15:33:16.927 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-18T15:33:16.934 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-18T15:33:17.013 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-18T15:33:17.018 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-18T15:33:17.102 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-18T15:33:17.104 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-18T15:33:17.144 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-18T15:33:17.187 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-18T15:33:17.232 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-03-18T15:33:17.232 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-03-18T15:33:17.244 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-18T15:33:17.332 DEBUG:teuthology.orchestra.run.smithi005:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-03-18T15:33:17.343 DEBUG:teuthology.orchestra.run.smithi080:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-03-18T15:33:17.357 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-18T15:33:17.451 DEBUG:teuthology.orchestra.run.smithi080:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-03-18T15:33:17.461 DEBUG:teuthology.orchestra.run.smithi080:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-03-18T15:33:17.515 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-18T15:33:17.604 DEBUG:teuthology.orchestra.run.smithi080:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-03-18T15:33:17.656 DEBUG:teuthology.orchestra.run.smithi005:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-18T15:33:17.659 DEBUG:teuthology.orchestra.run.smithi080:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-18T15:33:17.705 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-03-18T15:33:17.706 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-18T15:33:17.706 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-18T15:33:17.711 INFO:tasks.ceph.mon.a:Started 2024-03-18T15:33:17.711 INFO:tasks.ceph.mon.b:Restarting daemon 2024-03-18T15:33:17.711 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-03-18T15:33:17.751 INFO:tasks.ceph.mon.b:Started 2024-03-18T15:33:17.751 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-18T15:33:17.752 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-18T15:33:17.754 INFO:tasks.ceph.mon.c:Started 2024-03-18T15:33:17.755 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-03-18T15:33:17.755 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-03-18T15:33:17.755 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-03-18T15:33:17.757 INFO:tasks.ceph.mgr.y:Started 2024-03-18T15:33:17.757 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-03-18T15:33:17.758 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-03-18T15:33:17.760 INFO:tasks.ceph.mgr.x:Started 2024-03-18T15:33:17.760 INFO:tasks.ceph:Setting crush tunables to default 2024-03-18T15:33:17.760 DEBUG:teuthology.orchestra.run.smithi005:> sudo ceph --cluster ceph osd crush tunables default 2024-03-18T15:33:17.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:17.873+0000 7fa560360640 1 Processor -- start 2024-03-18T15:33:17.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:17.873+0000 7fa560360640 1 -- start start 2024-03-18T15:33:17.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:17.873+0000 7fa560360640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa55806d2d0 0x7fa5580b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:17.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:17.873+0000 7fa560360640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa55813f260 0x7fa55813f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:17.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:17.873+0000 7fa560360640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5580b0ea0 con 0x7fa5581453a0 2024-03-18T15:33:17.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:17.873+0000 7fa560360640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fa5580728c0 con 0x7fa55806d2d0 2024-03-18T15:33:17.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:17.873+0000 7fa560360640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fa55806d6b0 con 0x7fa55813f260 2024-03-18T15:33:17.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:17.873+0000 7fa55e8fd640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa5581453a0 0x7fa558145810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:35300/0 (socket says 172.21.15.5:35300) 2024-03-18T15:33:17.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:17.873+0000 7fa55e8fd640 1 -- 172.21.15.5:0/3735620054 learned_addr learned my addr 172.21.15.5:0/3735620054 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:33:17.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:17.873+0000 7fa55d8fb640 1 --2- 172.21.15.5:0/3735620054 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa55813f260 0x7fa55813f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:17.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:17.873+0000 7fa55e0fc640 1 --2- 172.21.15.5:0/3735620054 >> v2:172.21.15.80:3300/0 conn(0x7fa55806d2d0 0x7fa5580b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:17.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:17.873+0000 7fa55d8fb640 1 -- 172.21.15.5:0/3735620054 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa55813f260 msgr2=0x7fa55813f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-18T15:33:17.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:17.873+0000 7fa55d8fb640 1 -- 172.21.15.5:0/3735620054 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa55813f260 msgr2=0x7fa55813f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-18T15:33:17.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:17.873+0000 7fa55e0fc640 1 -- 172.21.15.5:0/3735620054 >> v2:172.21.15.80:3300/0 conn(0x7fa55806d2d0 msgr2=0x7fa5580b23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-18T15:33:17.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:17.873+0000 7fa55d8fb640 1 --2- 172.21.15.5:0/3735620054 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa55813f260 0x7fa55813f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-18T15:33:17.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:17.873+0000 7fa55e0fc640 1 -- 172.21.15.5:0/3735620054 >> v2:172.21.15.80:3300/0 conn(0x7fa55806d2d0 msgr2=0x7fa5580b23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-18T15:33:17.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:17.873+0000 7fa55e0fc640 1 --2- 172.21.15.5:0/3735620054 >> v2:172.21.15.80:3300/0 conn(0x7fa55806d2d0 0x7fa5580b23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-18T15:33:17.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:17.873+0000 7fa55d8fb640 1 --2- 172.21.15.5:0/3735620054 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa55813f260 0x7fa55813f6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T15:33:17.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:17.873+0000 7fa55e0fc640 1 --2- 172.21.15.5:0/3735620054 >> v2:172.21.15.80:3300/0 conn(0x7fa55806d2d0 0x7fa5580b23a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-18T15:33:18.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.077+0000 7fa55d8fb640 1 --2- 172.21.15.5:0/3735620054 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa55813f260 0x7fa55813f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:18.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.077+0000 7fa55e0fc640 1 --2- 172.21.15.5:0/3735620054 >> v2:172.21.15.80:3300/0 conn(0x7fa55806d2d0 0x7fa5580b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:18.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.077+0000 7fa55d8fb640 1 -- 172.21.15.5:0/3735620054 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa55813f260 msgr2=0x7fa55813f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-18T15:33:18.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.077+0000 7fa55d8fb640 1 -- 172.21.15.5:0/3735620054 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa55813f260 msgr2=0x7fa55813f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-18T15:33:18.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.077+0000 7fa55d8fb640 1 --2- 172.21.15.5:0/3735620054 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa55813f260 0x7fa55813f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-18T15:33:18.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.077+0000 7fa55d8fb640 1 --2- 172.21.15.5:0/3735620054 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa55813f260 0x7fa55813f6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-18T15:33:18.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.077+0000 7fa55e0fc640 1 -- 172.21.15.5:0/3735620054 >> v2:172.21.15.80:3300/0 conn(0x7fa55806d2d0 msgr2=0x7fa5580b23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-18T15:33:18.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.077+0000 7fa55e0fc640 1 -- 172.21.15.5:0/3735620054 >> v2:172.21.15.80:3300/0 conn(0x7fa55806d2d0 msgr2=0x7fa5580b23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-18T15:33:18.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.077+0000 7fa55e0fc640 1 --2- 172.21.15.5:0/3735620054 >> v2:172.21.15.80:3300/0 conn(0x7fa55806d2d0 0x7fa5580b23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-18T15:33:18.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.077+0000 7fa55e0fc640 1 --2- 172.21.15.5:0/3735620054 >> v2:172.21.15.80:3300/0 conn(0x7fa55806d2d0 0x7fa5580b23a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-18T15:33:18.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.117+0000 7fa55d0fa640 1 -- 172.21.15.5:0/3735620054 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2880366285 0 0) 0x7fa5580b0ea0 con 0x7fa5581453a0 2024-03-18T15:33:18.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.117+0000 7fa55d0fa640 1 -- 172.21.15.5:0/3735620054 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa544003530 con 0x7fa5581453a0 2024-03-18T15:33:18.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.117+0000 7fa55d0fa640 1 -- 172.21.15.5:0/3735620054 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 319517245 0 0) 0x7fa544003530 con 0x7fa5581453a0 2024-03-18T15:33:18.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.117+0000 7fa55d0fa640 1 -- 172.21.15.5:0/3735620054 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa55813f260 msgr2=0x7fa55813f6a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T15:33:18.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.117+0000 7fa55d0fa640 1 --2- 172.21.15.5:0/3735620054 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa55813f260 0x7fa55813f6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:18.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.117+0000 7fa55d0fa640 1 -- 172.21.15.5:0/3735620054 >> v2:172.21.15.80:3300/0 conn(0x7fa55806d2d0 msgr2=0x7fa5580b23a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T15:33:18.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.117+0000 7fa55d0fa640 1 --2- 172.21.15.5:0/3735620054 >> v2:172.21.15.80:3300/0 conn(0x7fa55806d2d0 0x7fa5580b23a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:18.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.117+0000 7fa55d0fa640 1 -- 172.21.15.5:0/3735620054 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5580b0bc0 con 0x7fa5581453a0 2024-03-18T15:33:18.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.117+0000 7fa55d0fa640 1 -- 172.21.15.5:0/3735620054 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3837549003 0 0) 0x7fa550001a00 con 0x7fa5581453a0 2024-03-18T15:33:18.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.117+0000 7fa55d0fa640 1 -- 172.21.15.5:0/3735620054 >> v1:172.21.15.5:6789/0 conn(0x7fa5581453a0 legacy=0x7fa558145810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:18.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.117+0000 7fa55d0fa640 1 --2- 172.21.15.5:0/3735620054 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa55813f260 0x7fa544003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:18.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.117+0000 7fa55d0fa640 1 --2- 172.21.15.5:0/3735620054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa544004120 0x7fa544004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:18.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.117+0000 7fa55d0fa640 1 --2- 172.21.15.5:0/3735620054 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa544004d50 0x7fa5440051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:18.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.117+0000 7fa55d0fa640 1 -- 172.21.15.5:0/3735620054 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa5580728c0 con 0x7fa544004120 2024-03-18T15:33:18.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.117+0000 7fa55d0fa640 1 -- 172.21.15.5:0/3735620054 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa55806d6b0 con 0x7fa544004d50 2024-03-18T15:33:18.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.117+0000 7fa55d0fa640 1 -- 172.21.15.5:0/3735620054 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fa54400bcb0 con 0x7fa55813f260 2024-03-18T15:33:18.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.117+0000 7fa55e8fd640 1 --2- 172.21.15.5:0/3735620054 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa544004d50 0x7fa5440051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:18.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.117+0000 7fa55d8fb640 1 --2- 172.21.15.5:0/3735620054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa544004120 0x7fa544004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:18.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.117+0000 7fa55e0fc640 1 --2- 172.21.15.5:0/3735620054 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa55813f260 0x7fa544003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:18.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.121+0000 7fa55d8fb640 1 -- 172.21.15.5:0/3735620054 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa55813f260 msgr2=0x7fa544003aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:18.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.121+0000 7fa55d8fb640 1 --2- 172.21.15.5:0/3735620054 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa55813f260 0x7fa544003aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:18.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.121+0000 7fa55d8fb640 1 -- 172.21.15.5:0/3735620054 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa544004d50 msgr2=0x7fa5440051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:18.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.121+0000 7fa55d8fb640 1 --2- 172.21.15.5:0/3735620054 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa544004d50 0x7fa5440051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:18.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.121+0000 7fa55d8fb640 1 -- 172.21.15.5:0/3735620054 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa544003530 con 0x7fa544004120 2024-03-18T15:33:18.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.121+0000 7fa55e0fc640 1 --2- 172.21.15.5:0/3735620054 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa55813f260 0x7fa544003aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:33:18.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.125+0000 7fa55d8fb640 1 --2- 172.21.15.5:0/3735620054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa544004120 0x7fa544004810 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa54800e8d0 tx=0x7fa54801c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79e24d5f0903bfba server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:18.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.125+0000 7fa55d0fa640 1 -- 172.21.15.5:0/3735620054 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5480ab070 con 0x7fa544004120 2024-03-18T15:33:18.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.125+0000 7fa55d0fa640 1 -- 172.21.15.5:0/3735620054 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5480a5740 con 0x7fa544004120 2024-03-18T15:33:18.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.125+0000 7fa55d0fa640 1 -- 172.21.15.5:0/3735620054 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa54800f040 con 0x7fa544004120 2024-03-18T15:33:18.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.125+0000 7fa560360640 1 -- 172.21.15.5:0/3735620054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa544004120 msgr2=0x7fa544004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:18.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.125+0000 7fa560360640 1 --2- 172.21.15.5:0/3735620054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa544004120 0x7fa544004810 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa54800e8d0 tx=0x7fa54801c840 comp rx=0 tx=0).stop 2024-03-18T15:33:18.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.125+0000 7fa55e8fd640 1 -- 172.21.15.5:0/3735620054 reap_dead start 2024-03-18T15:33:18.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.125+0000 7fa560360640 1 -- 172.21.15.5:0/3735620054 shutdown_connections 2024-03-18T15:33:18.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.125+0000 7fa560360640 1 -- 172.21.15.5:0/3735620054 >> 172.21.15.5:0/3735620054 conn(0x7fa5580b5670 msgr2=0x7fa5580b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:18.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.125+0000 7fa560360640 1 -- 172.21.15.5:0/3735620054 shutdown_connections 2024-03-18T15:33:18.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.125+0000 7fa560360640 1 -- 172.21.15.5:0/3735620054 wait complete. 2024-03-18T15:33:18.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.129+0000 7fa560360640 1 Processor -- start 2024-03-18T15:33:18.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.129+0000 7fa560360640 1 -- start start 2024-03-18T15:33:18.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.129+0000 7fa560360640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa5580b6bd0 0x7fa5580b6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:18.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.129+0000 7fa560360640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5580b84a0 0x7fa558136c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:18.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.133+0000 7fa560360640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa558134aa0 0x7fa558134f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:18.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.133+0000 7fa560360640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa5580b0960 con 0x7fa558134aa0 2024-03-18T15:33:18.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.133+0000 7fa560360640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa5580b07e0 con 0x7fa5580b84a0 2024-03-18T15:33:18.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.133+0000 7fa560360640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fa5580b5140 con 0x7fa5580b6bd0 2024-03-18T15:33:18.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.133+0000 7fa55e8fd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa558134aa0 0x7fa558134f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:18.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.133+0000 7fa55d8fb640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5580b84a0 0x7fa558136c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:18.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.133+0000 7fa55e8fd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa558134aa0 0x7fa558134f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60008/0 (socket says 172.21.15.5:60008) 2024-03-18T15:33:18.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.133+0000 7fa55e0fc640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa5580b6bd0 0x7fa5580b6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:18.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.133+0000 7fa55d8fb640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5580b84a0 0x7fa558136c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:55688/0 (socket says 172.21.15.5:55688) 2024-03-18T15:33:18.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.133+0000 7fa55e8fd640 1 -- 172.21.15.5:0/406387402 learned_addr learned my addr 172.21.15.5:0/406387402 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:18.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.133+0000 7fa55e0fc640 1 -- 172.21.15.5:0/406387402 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5580b84a0 msgr2=0x7fa558136c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:18.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.133+0000 7fa55e0fc640 1 --2- 172.21.15.5:0/406387402 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5580b84a0 0x7fa558136c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:18.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.133+0000 7fa55e0fc640 1 -- 172.21.15.5:0/406387402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa558134aa0 msgr2=0x7fa558134f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:18.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.133+0000 7fa55e0fc640 1 --2- 172.21.15.5:0/406387402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa558134aa0 0x7fa558134f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:18.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.133+0000 7fa55e0fc640 1 -- 172.21.15.5:0/406387402 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5580b78c0 con 0x7fa5580b6bd0 2024-03-18T15:33:18.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.133+0000 7fa55e0fc640 1 --2- 172.21.15.5:0/406387402 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa5580b6bd0 0x7fa5580b6fb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa554013740 tx=0x7fa554013c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:18.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.133+0000 7fa54effd640 1 -- 172.21.15.5:0/406387402 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55401a070 con 0x7fa5580b6bd0 2024-03-18T15:33:18.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.133+0000 7fa54effd640 1 -- 172.21.15.5:0/406387402 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa554004440 con 0x7fa5580b6bd0 2024-03-18T15:33:18.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.133+0000 7fa54effd640 1 -- 172.21.15.5:0/406387402 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa554011a60 con 0x7fa5580b6bd0 2024-03-18T15:33:18.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.133+0000 7fa560360640 1 -- 172.21.15.5:0/406387402 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa558136730 con 0x7fa5580b6bd0 2024-03-18T15:33:18.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.133+0000 7fa560360640 1 -- 172.21.15.5:0/406387402 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa558137600 con 0x7fa5580b6bd0 2024-03-18T15:33:18.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.133+0000 7fa54effd640 1 -- 172.21.15.5:0/406387402 <== mon.2 v2:172.21.15.80:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa55401d070 con 0x7fa5580b6bd0 2024-03-18T15:33:18.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.133+0000 7fa54effd640 1 -- 172.21.15.5:0/406387402 <== mon.2 v2:172.21.15.80:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa5540167a0 con 0x7fa5580b6bd0 2024-03-18T15:33:18.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.133+0000 7fa560360640 1 -- 172.21.15.5:0/406387402 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5580b76d0 con 0x7fa5580b6bd0 2024-03-18T15:33:18.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.149+0000 7fa54effd640 1 -- 172.21.15.5:0/406387402 <== mon.2 v2:172.21.15.80:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fa554003df0 con 0x7fa5580b6bd0 2024-03-18T15:33:18.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.213+0000 7fa560360640 1 -- 172.21.15.5:0/406387402 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fa55804f550 con 0x7fa5580b6bd0 2024-03-18T15:33:18.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.225+0000 7fa54effd640 1 -- 172.21.15.5:0/406387402 <== mon.2 v2:172.21.15.80:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7fa554011c00 con 0x7fa5580b6bd0 2024-03-18T15:33:18.226 INFO:teuthology.orchestra.run.smithi005.stderr:adjusted tunables profile to default 2024-03-18T15:33:18.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.225+0000 7fa560360640 1 -- 172.21.15.5:0/406387402 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa5580b6bd0 msgr2=0x7fa5580b6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:18.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.225+0000 7fa560360640 1 --2- 172.21.15.5:0/406387402 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa5580b6bd0 0x7fa5580b6fb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa554013740 tx=0x7fa554013c00 comp rx=0 tx=0).stop 2024-03-18T15:33:18.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.225+0000 7fa560360640 1 -- 172.21.15.5:0/406387402 shutdown_connections 2024-03-18T15:33:18.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.225+0000 7fa560360640 1 --2- 172.21.15.5:0/406387402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa558134aa0 0x7fa558134f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:18.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.225+0000 7fa560360640 1 --2- 172.21.15.5:0/406387402 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5580b84a0 0x7fa558136c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:18.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.225+0000 7fa560360640 1 --2- 172.21.15.5:0/406387402 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa5580b6bd0 0x7fa5580b6fb0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:18.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.225+0000 7fa560360640 1 -- 172.21.15.5:0/406387402 >> 172.21.15.5:0/406387402 conn(0x7fa5580b5670 msgr2=0x7fa55814ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:18.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.225+0000 7fa560360640 1 -- 172.21.15.5:0/406387402 shutdown_connections 2024-03-18T15:33:18.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:18.225+0000 7fa560360640 1 -- 172.21.15.5:0/406387402 wait complete. 2024-03-18T15:33:18.246 INFO:tasks.ceph:check_enable_crimson: False 2024-03-18T15:33:18.246 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-03-18T15:33:18.246 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-03-18T15:33:18.246 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-03-18T15:33:18.259 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-03-18T15:33:18.259 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-03-18T15:33:18.312 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-03-18T15:33:18.312 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-03-18T15:33:18.327 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-03-18T15:33:18.325+0000 7fb3a3d19180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-18T15:33:18.339 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-03-18T15:33:18.339 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-03-18T15:33:18.394 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-03-18T15:33:18.394 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-03-18T15:33:18.400 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-03-18T15:33:18.393+0000 7f43fab64180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-18T15:33:18.407 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-03-18T15:33:18.407 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-03-18T15:33:18.460 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-03-18T15:33:18.460 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-03-18T15:33:18.512 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-03-18T15:33:18.512 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-03-18T15:33:18.564 DEBUG:teuthology.orchestra.run.smithi080:> sudo ceph --cluster ceph osd new df1c754b-af59-4d16-8919-fbb6386b3c48 0 2024-03-18T15:33:18.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.725+0000 7ff20148f640 1 Processor -- start 2024-03-18T15:33:18.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.725+0000 7ff20148f640 1 -- start start 2024-03-18T15:33:18.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.725+0000 7ff20148f640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff1fc0b6c60 0x7ff1fc0b45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:18.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.725+0000 7ff20148f640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff1fc148f80 0x7ff1fc14fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:18.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.725+0000 7ff20148f640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1fc0b6020 con 0x7ff1fc072b20 2024-03-18T15:33:18.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.725+0000 7ff20148f640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7ff1fc0b79b0 con 0x7ff1fc148f80 2024-03-18T15:33:18.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.725+0000 7ff20148f640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7ff1fc0b76a0 con 0x7ff1fc0b6c60 2024-03-18T15:33:18.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.725+0000 7ff1faffd640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff1fc0b6c60 0x7ff1fc0b45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:18.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.725+0000 7ff1fa7fc640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7ff1fc072b20 0x7ff1fc148870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:50818/0 (socket says 172.21.15.80:50818) 2024-03-18T15:33:18.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.725+0000 7ff1fa7fc640 1 -- 172.21.15.80:0/3347770494 learned_addr learned my addr 172.21.15.80:0/3347770494 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-03-18T15:33:18.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.725+0000 7ff1faffd640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff1fc0b6c60 0x7ff1fc0b45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.80:52324/0 (socket says 172.21.15.80:52324) 2024-03-18T15:33:18.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.725+0000 7ff1fb7fe640 1 --2- 172.21.15.80:0/3347770494 >> v2:172.21.15.5:3300/0 conn(0x7ff1fc148f80 0x7ff1fc14fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:18.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.725+0000 7ff1f9ffb640 1 -- 172.21.15.80:0/3347770494 <== mon.1 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3136287560 0 0) 0x7ff1fc0b6020 con 0x7ff1fc072b20 2024-03-18T15:33:18.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.725+0000 7ff1f9ffb640 1 -- 172.21.15.80:0/3347770494 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff1e4003530 con 0x7ff1fc072b20 2024-03-18T15:33:18.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.725+0000 7ff1faffd640 1 -- 172.21.15.80:0/3347770494 >> v2:172.21.15.5:3300/0 conn(0x7ff1fc148f80 msgr2=0x7ff1fc14fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:18.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.725+0000 7ff1faffd640 1 --2- 172.21.15.80:0/3347770494 >> v2:172.21.15.5:3300/0 conn(0x7ff1fc148f80 0x7ff1fc14fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:18.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.725+0000 7ff1faffd640 1 -- 172.21.15.80:0/3347770494 >> v1:172.21.15.80:6789/0 conn(0x7ff1fc072b20 legacy=0x7ff1fc148870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:18.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.725+0000 7ff1faffd640 1 -- 172.21.15.80:0/3347770494 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1fc0b7040 con 0x7ff1fc0b6c60 2024-03-18T15:33:18.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.729+0000 7ff1faffd640 1 --2- 172.21.15.80:0/3347770494 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff1fc0b6c60 0x7ff1fc0b45c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4009d90 tx=0x7ff1f401c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7d5d5016fd21048 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:18.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.729+0000 7ff1f9ffb640 1 -- 172.21.15.80:0/3347770494 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f40b2070 con 0x7ff1fc0b6c60 2024-03-18T15:33:18.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.729+0000 7ff1f9ffb640 1 -- 172.21.15.80:0/3347770494 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1f40a59c0 con 0x7ff1fc0b6c60 2024-03-18T15:33:18.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.729+0000 7ff1f9ffb640 1 -- 172.21.15.80:0/3347770494 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f40a5d90 con 0x7ff1fc0b6c60 2024-03-18T15:33:18.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.729+0000 7ff20148f640 1 -- 172.21.15.80:0/3347770494 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff1fc0b6c60 msgr2=0x7ff1fc0b45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:18.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.729+0000 7ff20148f640 1 --2- 172.21.15.80:0/3347770494 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff1fc0b6c60 0x7ff1fc0b45c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff1f4009d90 tx=0x7ff1f401c8e0 comp rx=0 tx=0).stop 2024-03-18T15:33:18.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.729+0000 7ff20148f640 1 -- 172.21.15.80:0/3347770494 shutdown_connections 2024-03-18T15:33:18.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.729+0000 7ff20148f640 1 --2- 172.21.15.80:0/3347770494 >> v2:172.21.15.5:3300/0 conn(0x7ff1fc148f80 0x7ff1fc14fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:18.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.729+0000 7ff20148f640 1 --2- 172.21.15.80:0/3347770494 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff1fc0b6c60 0x7ff1fc0b45c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:18.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.729+0000 7ff20148f640 1 -- 172.21.15.80:0/3347770494 >> 172.21.15.80:0/3347770494 conn(0x7ff1fc066fc0 msgr2=0x7ff1fc0673d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:18.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.729+0000 7ff20148f640 1 -- 172.21.15.80:0/3347770494 shutdown_connections 2024-03-18T15:33:18.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.729+0000 7ff20148f640 1 -- 172.21.15.80:0/3347770494 wait complete. 2024-03-18T15:33:18.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.729+0000 7ff20148f640 1 Processor -- start 2024-03-18T15:33:18.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.729+0000 7ff20148f640 1 -- start start 2024-03-18T15:33:18.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.729+0000 7ff20148f640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff1fc072b20 0x7ff1fc140840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:18.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.729+0000 7ff20148f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1fc0b6c60 0x7ff1fc14e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:18.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.729+0000 7ff20148f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1fc1429d0 0x7ff1fc14eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:18.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.729+0000 7ff20148f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc152c90 con 0x7ff1fc0b6c60 2024-03-18T15:33:18.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.729+0000 7ff20148f640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc152b10 con 0x7ff1fc1429d0 2024-03-18T15:33:18.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.729+0000 7ff20148f640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7ff1fc152e10 con 0x7ff1fc072b20 2024-03-18T15:33:18.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.733+0000 7ff1faffd640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff1fc072b20 0x7ff1fc140840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:18.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.733+0000 7ff1fb7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1fc1429d0 0x7ff1fc14eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:18.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.733+0000 7ff1faffd640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff1fc072b20 0x7ff1fc140840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.80:52332/0 (socket says 172.21.15.80:52332) 2024-03-18T15:33:18.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.733+0000 7ff1fb7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1fc1429d0 0x7ff1fc14eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50416/0 (socket says 172.21.15.80:50416) 2024-03-18T15:33:18.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.733+0000 7ff1faffd640 1 -- 172.21.15.80:0/2620496168 learned_addr learned my addr 172.21.15.80:0/2620496168 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-03-18T15:33:18.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.733+0000 7ff1fa7fc640 1 --2- 172.21.15.80:0/2620496168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1fc0b6c60 0x7ff1fc14e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:18.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.733+0000 7ff1fb7fe640 1 -- 172.21.15.80:0/2620496168 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff1fc072b20 msgr2=0x7ff1fc140840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:18.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.733+0000 7ff1fb7fe640 1 --2- 172.21.15.80:0/2620496168 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff1fc072b20 0x7ff1fc140840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:18.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.733+0000 7ff1fb7fe640 1 -- 172.21.15.80:0/2620496168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1fc0b6c60 msgr2=0x7ff1fc14e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:18.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.733+0000 7ff1fb7fe640 1 --2- 172.21.15.80:0/2620496168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1fc0b6c60 0x7ff1fc14e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:18.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.733+0000 7ff1fb7fe640 1 -- 172.21.15.80:0/2620496168 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1fc0a9720 con 0x7ff1fc1429d0 2024-03-18T15:33:18.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.733+0000 7ff1faffd640 1 --2- 172.21.15.80:0/2620496168 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff1fc072b20 0x7ff1fc140840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:33:18.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.733+0000 7ff1fb7fe640 1 --2- 172.21.15.80:0/2620496168 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1fc1429d0 0x7ff1fc14eb90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec00e7c0 tx=0x7ff1ec00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:18.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.733+0000 7ff1e37fe640 1 -- 172.21.15.80:0/2620496168 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec017070 con 0x7ff1fc1429d0 2024-03-18T15:33:18.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.733+0000 7ff1e37fe640 1 -- 172.21.15.80:0/2620496168 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1ec00c730 con 0x7ff1fc1429d0 2024-03-18T15:33:18.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.733+0000 7ff20148f640 1 -- 172.21.15.80:0/2620496168 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1fc141de0 con 0x7ff1fc1429d0 2024-03-18T15:33:18.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.733+0000 7ff1e37fe640 1 -- 172.21.15.80:0/2620496168 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec00cb00 con 0x7ff1fc1429d0 2024-03-18T15:33:18.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.733+0000 7ff20148f640 1 -- 172.21.15.80:0/2620496168 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1fc142080 con 0x7ff1fc1429d0 2024-03-18T15:33:18.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.733+0000 7ff1e37fe640 1 -- 172.21.15.80:0/2620496168 <== mon.1 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff1ec01a070 con 0x7ff1fc1429d0 2024-03-18T15:33:18.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.733+0000 7ff1e37fe640 1 -- 172.21.15.80:0/2620496168 <== mon.1 v2:172.21.15.80:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7ff1ec003960 con 0x7ff1fc1429d0 2024-03-18T15:33:18.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.733+0000 7ff20148f640 1 -- 172.21.15.80:0/2620496168 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1fc04f550 con 0x7ff1fc1429d0 2024-03-18T15:33:18.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.737+0000 7ff1e37fe640 1 -- 172.21.15.80:0/2620496168 <== mon.1 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7ff1fc0a9720 con 0x7ff1fc1429d0 2024-03-18T15:33:18.805 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-03-18T15:33:18.801+0000 7fb3a3d19180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-18T15:33:18.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.809+0000 7ff20148f640 1 -- 172.21.15.80:0/2620496168 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd new", "uuid": "df1c754b-af59-4d16-8919-fbb6386b3c48", "id": 0} v 0) -- 0x7ff1fc0b2950 con 0x7ff1fc1429d0 2024-03-18T15:33:18.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.817+0000 7ff1e37fe640 1 -- 172.21.15.80:0/2620496168 <== mon.1 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "df1c754b-af59-4d16-8919-fbb6386b3c48", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7ff1ec00cd40 con 0x7ff1fc1429d0 2024-03-18T15:33:18.823 INFO:teuthology.orchestra.run.smithi080.stdout:0 2024-03-18T15:33:18.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.817+0000 7ff20148f640 1 -- 172.21.15.80:0/2620496168 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1fc1429d0 msgr2=0x7ff1fc14eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:18.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.817+0000 7ff20148f640 1 --2- 172.21.15.80:0/2620496168 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1fc1429d0 0x7ff1fc14eb90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec00e7c0 tx=0x7ff1ec00ec80 comp rx=0 tx=0).stop 2024-03-18T15:33:18.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.817+0000 7ff20148f640 1 -- 172.21.15.80:0/2620496168 shutdown_connections 2024-03-18T15:33:18.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.817+0000 7ff20148f640 1 --2- 172.21.15.80:0/2620496168 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1fc1429d0 0x7ff1fc14eb90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:18.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.817+0000 7ff20148f640 1 --2- 172.21.15.80:0/2620496168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff1fc0b6c60 0x7ff1fc14e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:18.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.817+0000 7ff20148f640 1 --2- 172.21.15.80:0/2620496168 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff1fc072b20 0x7ff1fc140840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:18.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.817+0000 7ff20148f640 1 -- 172.21.15.80:0/2620496168 >> 172.21.15.80:0/2620496168 conn(0x7ff1fc066fc0 msgr2=0x7ff1fc071b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:18.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.817+0000 7ff20148f640 1 -- 172.21.15.80:0/2620496168 shutdown_connections 2024-03-18T15:33:18.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.817+0000 7ff20148f640 1 -- 172.21.15.80:0/2620496168 wait complete. 2024-03-18T15:33:18.841 DEBUG:teuthology.orchestra.run.smithi080:> sudo ceph --cluster ceph osd new b9a08b5e-8af4-4144-bdba-d69ed1ac627a 1 2024-03-18T15:33:18.897 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-03-18T15:33:18.893+0000 7f43fab64180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-18T15:33:18.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.953+0000 7f047c951640 1 Processor -- start 2024-03-18T15:33:18.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.953+0000 7f047c951640 1 -- start start 2024-03-18T15:33:18.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.953+0000 7f047c951640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f047806b4e0 0x7f04781412d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:18.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.953+0000 7f047c951640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f047806b8c0 con 0x7f0478147d80 2024-03-18T15:33:18.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.953+0000 7f047c951640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f047806ef40 con 0x7f0478141810 2024-03-18T15:33:18.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.953+0000 7f047c951640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f047806e2d0 con 0x7f047806b4e0 2024-03-18T15:33:18.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.953+0000 7f0475d9b640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f0478141810 0x7f0478147670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:50832/0 (socket says 172.21.15.80:50832) 2024-03-18T15:33:18.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.953+0000 7f047659c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f047806b4e0 0x7f04781412d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:18.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.953+0000 7f0475d9b640 1 -- 172.21.15.80:0/4219924765 learned_addr learned my addr 172.21.15.80:0/4219924765 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-03-18T15:33:18.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.953+0000 7f047659c640 1 -- 172.21.15.80:0/4219924765 >> v1:172.21.15.80:6789/0 conn(0x7f0478141810 legacy=0x7f0478147670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:18.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.953+0000 7f047659c640 1 -- 172.21.15.80:0/4219924765 >> v1:172.21.15.5:6789/0 conn(0x7f0478147d80 legacy=0x7f0478151ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:18.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.953+0000 7f047659c640 1 -- 172.21.15.80:0/4219924765 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0478070390 con 0x7f047806b4e0 2024-03-18T15:33:18.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.953+0000 7f047659c640 1 --2- 172.21.15.80:0/4219924765 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f047806b4e0 0x7f04781412d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f04680097c0 tx=0x7f046801c3d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8b7f8df098bdeb8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:18.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.953+0000 7f047559a640 1 -- 172.21.15.80:0/4219924765 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04680b0070 con 0x7f047806b4e0 2024-03-18T15:33:18.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.953+0000 7f047559a640 1 -- 172.21.15.80:0/4219924765 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04680ab600 con 0x7f047806b4e0 2024-03-18T15:33:18.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.953+0000 7f047559a640 1 -- 172.21.15.80:0/4219924765 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04680a4e60 con 0x7f047806b4e0 2024-03-18T15:33:18.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.953+0000 7f047c951640 1 -- 172.21.15.80:0/4219924765 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f047806b4e0 msgr2=0x7f04781412d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:18.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.953+0000 7f047c951640 1 --2- 172.21.15.80:0/4219924765 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f047806b4e0 0x7f04781412d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f04680097c0 tx=0x7f046801c3d0 comp rx=0 tx=0).stop 2024-03-18T15:33:18.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.953+0000 7f047c951640 1 -- 172.21.15.80:0/4219924765 shutdown_connections 2024-03-18T15:33:18.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.953+0000 7f047c951640 1 --2- 172.21.15.80:0/4219924765 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f047806b4e0 0x7f04781412d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:18.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.953+0000 7f047c951640 1 -- 172.21.15.80:0/4219924765 >> 172.21.15.80:0/4219924765 conn(0x7f04780b79f0 msgr2=0x7f04780b7dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:18.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.953+0000 7f047c951640 1 -- 172.21.15.80:0/4219924765 shutdown_connections 2024-03-18T15:33:18.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.953+0000 7f047c951640 1 -- 172.21.15.80:0/4219924765 wait complete. 2024-03-18T15:33:18.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.953+0000 7f047c951640 1 Processor -- start 2024-03-18T15:33:18.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.953+0000 7f047c951640 1 -- start start 2024-03-18T15:33:18.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.953+0000 7f047c951640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f047806b4e0 0x7f04781465e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:18.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.953+0000 7f047c951640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0478141810 0x7f0478148ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:18.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.953+0000 7f047c951640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0478147d80 0x7f0478149020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:18.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.953+0000 7f047c951640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f047806e8a0 con 0x7f0478147d80 2024-03-18T15:33:18.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.953+0000 7f047c951640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f047806e720 con 0x7f047806b4e0 2024-03-18T15:33:18.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.953+0000 7f047c951640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f0478155c20 con 0x7f0478141810 2024-03-18T15:33:18.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.961+0000 7f0475d9b640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0478141810 0x7f0478148ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:18.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.961+0000 7f047659c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f047806b4e0 0x7f04781465e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:18.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.961+0000 7f0475d9b640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0478141810 0x7f0478148ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.80:52348/0 (socket says 172.21.15.80:52348) 2024-03-18T15:33:18.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.961+0000 7f0475d9b640 1 -- 172.21.15.80:0/923007610 learned_addr learned my addr 172.21.15.80:0/923007610 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-03-18T15:33:18.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.961+0000 7f047659c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f047806b4e0 0x7f04781465e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50422/0 (socket says 172.21.15.80:50422) 2024-03-18T15:33:18.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.961+0000 7f0476d9d640 1 --2- 172.21.15.80:0/923007610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0478147d80 0x7f0478149020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:18.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.961+0000 7f0475d9b640 1 -- 172.21.15.80:0/923007610 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f047806b4e0 msgr2=0x7f04781465e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:18.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.961+0000 7f0475d9b640 1 --2- 172.21.15.80:0/923007610 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f047806b4e0 0x7f04781465e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:18.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.961+0000 7f0475d9b640 1 -- 172.21.15.80:0/923007610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0478147d80 msgr2=0x7f0478149020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:18.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.961+0000 7f0475d9b640 1 --2- 172.21.15.80:0/923007610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0478147d80 0x7f0478149020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:18.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.961+0000 7f0475d9b640 1 -- 172.21.15.80:0/923007610 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f047814a7c0 con 0x7f0478141810 2024-03-18T15:33:18.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.961+0000 7f047659c640 1 --2- 172.21.15.80:0/923007610 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f047806b4e0 0x7f04781465e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:33:18.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.961+0000 7f0475d9b640 1 --2- 172.21.15.80:0/923007610 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0478141810 0x7f0478148ae0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f046c00c450 tx=0x7f046c00c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:18.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.961+0000 7f0466ffd640 1 -- 172.21.15.80:0/923007610 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046c017070 con 0x7f0478141810 2024-03-18T15:33:18.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.961+0000 7f0466ffd640 1 -- 172.21.15.80:0/923007610 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f046c005cf0 con 0x7f0478141810 2024-03-18T15:33:18.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.961+0000 7f047c951640 1 -- 172.21.15.80:0/923007610 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f047814af30 con 0x7f0478141810 2024-03-18T15:33:18.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.961+0000 7f0466ffd640 1 -- 172.21.15.80:0/923007610 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046c007290 con 0x7f0478141810 2024-03-18T15:33:18.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.961+0000 7f047c951640 1 -- 172.21.15.80:0/923007610 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f047814b4f0 con 0x7f0478141810 2024-03-18T15:33:18.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.961+0000 7f0466ffd640 1 -- 172.21.15.80:0/923007610 <== mon.2 v2:172.21.15.80:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f046c01a070 con 0x7f0478141810 2024-03-18T15:33:18.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.961+0000 7f0466ffd640 1 -- 172.21.15.80:0/923007610 <== mon.2 v2:172.21.15.80:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f046c0127b0 con 0x7f0478141810 2024-03-18T15:33:18.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.961+0000 7f047c951640 1 -- 172.21.15.80:0/923007610 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f047804f550 con 0x7f0478141810 2024-03-18T15:33:18.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:18.961+0000 7f0466ffd640 1 -- 172.21.15.80:0/923007610 <== mon.2 v2:172.21.15.80:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f046c021020 con 0x7f0478141810 2024-03-18T15:33:19.008 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-03-18T15:33:19.005+0000 7fb3a3d19180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-18T15:33:19.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.033+0000 7f047c951640 1 -- 172.21.15.80:0/923007610 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b9a08b5e-8af4-4144-bdba-d69ed1ac627a", "id": 1} v 0) -- 0x7f0478074bf0 con 0x7f0478141810 2024-03-18T15:33:19.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.041+0000 7f0466ffd640 1 -- 172.21.15.80:0/923007610 <== mon.2 v2:172.21.15.80:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b9a08b5e-8af4-4144-bdba-d69ed1ac627a", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f046c0074f0 con 0x7f0478141810 2024-03-18T15:33:19.044 INFO:teuthology.orchestra.run.smithi080.stdout:1 2024-03-18T15:33:19.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.041+0000 7f047c951640 1 -- 172.21.15.80:0/923007610 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0478141810 msgr2=0x7f0478148ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:19.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.041+0000 7f047c951640 1 --2- 172.21.15.80:0/923007610 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0478141810 0x7f0478148ae0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f046c00c450 tx=0x7f046c00c910 comp rx=0 tx=0).stop 2024-03-18T15:33:19.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.041+0000 7f047c951640 1 -- 172.21.15.80:0/923007610 shutdown_connections 2024-03-18T15:33:19.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.041+0000 7f047c951640 1 --2- 172.21.15.80:0/923007610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0478147d80 0x7f0478149020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:19.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.041+0000 7f047c951640 1 --2- 172.21.15.80:0/923007610 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0478141810 0x7f0478148ae0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:19.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.041+0000 7f047c951640 1 --2- 172.21.15.80:0/923007610 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f047806b4e0 0x7f04781465e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:19.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.041+0000 7f047c951640 1 -- 172.21.15.80:0/923007610 >> 172.21.15.80:0/923007610 conn(0x7f04780b79f0 msgr2=0x7f047806d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:19.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.041+0000 7f047c951640 1 -- 172.21.15.80:0/923007610 shutdown_connections 2024-03-18T15:33:19.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.045+0000 7f047c951640 1 -- 172.21.15.80:0/923007610 wait complete. 2024-03-18T15:33:19.060 DEBUG:teuthology.orchestra.run.smithi080:> sudo ceph --cluster ceph osd new c3e8bb76-e382-439d-b380-62ef92f1255a 2 2024-03-18T15:33:19.107 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-03-18T15:33:19.105+0000 7f43fab64180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-18T15:33:19.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.181+0000 7f7aae0d8640 1 Processor -- start 2024-03-18T15:33:19.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.181+0000 7f7aae0d8640 1 -- start start 2024-03-18T15:33:19.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.181+0000 7f7aae0d8640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f7aa80afd40 0x7f7aa80b2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:19.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.181+0000 7f7aae0d8640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f7aa8143820 0x7f7aa813d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:19.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.185+0000 7f7aae0d8640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7aa80ba060 con 0x7f7aa813dac0 2024-03-18T15:33:19.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.185+0000 7f7aae0d8640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f7aa8143c80 con 0x7f7aa8143820 2024-03-18T15:33:19.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.185+0000 7f7aae0d8640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f7aa80b0120 con 0x7f7aa80afd40 2024-03-18T15:33:19.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.185+0000 7f7aa77fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f7aa80afd40 0x7f7aa80b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:19.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.185+0000 7f7aa7fff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f7aa813dac0 0x7f7aa80b0630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:50838/0 (socket says 172.21.15.80:50838) 2024-03-18T15:33:19.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.185+0000 7f7aa77fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f7aa80afd40 0x7f7aa80b2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50438/0 (socket says 172.21.15.80:50438) 2024-03-18T15:33:19.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.185+0000 7f7aa7fff640 1 -- 172.21.15.80:0/1898366177 learned_addr learned my addr 172.21.15.80:0/1898366177 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-03-18T15:33:19.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.185+0000 7f7aa6ffd640 1 --2- 172.21.15.80:0/1898366177 >> v2:172.21.15.5:3300/0 conn(0x7f7aa8143820 0x7f7aa813d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:19.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.185+0000 7f7aa77fe640 1 -- 172.21.15.80:0/1898366177 >> v2:172.21.15.5:3300/0 conn(0x7f7aa8143820 msgr2=0x7f7aa813d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:19.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.185+0000 7f7aa77fe640 1 --2- 172.21.15.80:0/1898366177 >> v2:172.21.15.5:3300/0 conn(0x7f7aa8143820 0x7f7aa813d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:19.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.185+0000 7f7aa77fe640 1 -- 172.21.15.80:0/1898366177 >> v1:172.21.15.80:6789/0 conn(0x7f7aa813dac0 legacy=0x7f7aa80b0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:19.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.185+0000 7f7aa67fc640 1 -- 172.21.15.80:0/1898366177 <== mon.1 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1455057049 0 0) 0x7f7aa80ba060 con 0x7f7aa813dac0 2024-03-18T15:33:19.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.185+0000 7f7aa77fe640 1 -- 172.21.15.80:0/1898366177 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aa8143eb0 con 0x7f7aa80afd40 2024-03-18T15:33:19.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.185+0000 7f7aa77fe640 1 --2- 172.21.15.80:0/1898366177 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7aa80afd40 0x7f7aa80b2280 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7aa80b6d10 tx=0x7f7aa001ea70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f4f88791fd5bc85 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:19.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.185+0000 7f7aa67fc640 1 -- 172.21.15.80:0/1898366177 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa000b2d0 con 0x7f7aa80afd40 2024-03-18T15:33:19.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.185+0000 7f7aa67fc640 1 -- 172.21.15.80:0/1898366177 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7aa00a78c0 con 0x7f7aa80afd40 2024-03-18T15:33:19.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.185+0000 7f7aa67fc640 1 -- 172.21.15.80:0/1898366177 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa00a7c90 con 0x7f7aa80afd40 2024-03-18T15:33:19.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.185+0000 7f7aae0d8640 1 -- 172.21.15.80:0/1898366177 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7aa80afd40 msgr2=0x7f7aa80b2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:19.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.185+0000 7f7aae0d8640 1 --2- 172.21.15.80:0/1898366177 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7aa80afd40 0x7f7aa80b2280 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7aa80b6d10 tx=0x7f7aa001ea70 comp rx=0 tx=0).stop 2024-03-18T15:33:19.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.185+0000 7f7aae0d8640 1 -- 172.21.15.80:0/1898366177 shutdown_connections 2024-03-18T15:33:19.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.185+0000 7f7aae0d8640 1 --2- 172.21.15.80:0/1898366177 >> v2:172.21.15.5:3300/0 conn(0x7f7aa8143820 0x7f7aa813d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:19.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.185+0000 7f7aae0d8640 1 --2- 172.21.15.80:0/1898366177 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7aa80afd40 0x7f7aa80b2280 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:19.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.185+0000 7f7aae0d8640 1 -- 172.21.15.80:0/1898366177 >> 172.21.15.80:0/1898366177 conn(0x7f7aa80b9190 msgr2=0x7f7aa80b95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:19.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.185+0000 7f7aae0d8640 1 -- 172.21.15.80:0/1898366177 shutdown_connections 2024-03-18T15:33:19.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.185+0000 7f7aae0d8640 1 -- 172.21.15.80:0/1898366177 wait complete. 2024-03-18T15:33:19.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.189+0000 7f7aae0d8640 1 Processor -- start 2024-03-18T15:33:19.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.189+0000 7f7aae0d8640 1 -- start start 2024-03-18T15:33:19.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.189+0000 7f7aae0d8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7aa80afd40 0x7f7aa80ae2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:19.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.189+0000 7f7aae0d8640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7aa813dac0 0x7f7aa81e2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:19.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.189+0000 7f7aae0d8640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7aa8143820 0x7f7aa812fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:19.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.189+0000 7f7aae0d8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa8150330 con 0x7f7aa80afd40 2024-03-18T15:33:19.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.189+0000 7f7aae0d8640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa81501b0 con 0x7f7aa8143820 2024-03-18T15:33:19.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.189+0000 7f7aae0d8640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f7aa81504b0 con 0x7f7aa813dac0 2024-03-18T15:33:19.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.189+0000 7f7aa7fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7aa8143820 0x7f7aa812fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:19.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.189+0000 7f7aa6ffd640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7aa813dac0 0x7f7aa81e2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:19.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.189+0000 7f7aa7fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7aa8143820 0x7f7aa812fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50448/0 (socket says 172.21.15.80:50448) 2024-03-18T15:33:19.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.189+0000 7f7aa7fff640 1 -- 172.21.15.80:0/1533379486 learned_addr learned my addr 172.21.15.80:0/1533379486 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-03-18T15:33:19.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.189+0000 7f7aa77fe640 1 --2- 172.21.15.80:0/1533379486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7aa80afd40 0x7f7aa80ae2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:19.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.189+0000 7f7aa7fff640 1 -- 172.21.15.80:0/1533379486 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7aa813dac0 msgr2=0x7f7aa81e2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:19.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.189+0000 7f7aa7fff640 1 --2- 172.21.15.80:0/1533379486 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7aa813dac0 0x7f7aa81e2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:19.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.189+0000 7f7aa7fff640 1 -- 172.21.15.80:0/1533379486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7aa80afd40 msgr2=0x7f7aa80ae2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:19.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.189+0000 7f7aa7fff640 1 --2- 172.21.15.80:0/1533379486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7aa80afd40 0x7f7aa80ae2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:19.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.189+0000 7f7aa7fff640 1 -- 172.21.15.80:0/1533379486 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aa80ae7e0 con 0x7f7aa8143820 2024-03-18T15:33:19.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.189+0000 7f7aa77fe640 1 --2- 172.21.15.80:0/1533379486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7aa80afd40 0x7f7aa80ae2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:33:19.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.189+0000 7f7aa6ffd640 1 --2- 172.21.15.80:0/1533379486 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7aa813dac0 0x7f7aa81e2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:33:19.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.189+0000 7f7aa7fff640 1 --2- 172.21.15.80:0/1533379486 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7aa8143820 0x7f7aa812fec0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7a9800c4b0 tx=0x7f7a9800c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:19.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.189+0000 7f7a9ffff640 1 -- 172.21.15.80:0/1533379486 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a98018070 con 0x7f7aa8143820 2024-03-18T15:33:19.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.189+0000 7f7a9ffff640 1 -- 172.21.15.80:0/1533379486 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a980074e0 con 0x7f7aa8143820 2024-03-18T15:33:19.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.189+0000 7f7a9ffff640 1 -- 172.21.15.80:0/1533379486 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a98004a30 con 0x7f7aa8143820 2024-03-18T15:33:19.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.189+0000 7f7aae0d8640 1 -- 172.21.15.80:0/1533379486 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aa81360b0 con 0x7f7aa8143820 2024-03-18T15:33:19.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.189+0000 7f7aae0d8640 1 -- 172.21.15.80:0/1533379486 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aa8134e20 con 0x7f7aa8143820 2024-03-18T15:33:19.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.189+0000 7f7a9ffff640 1 -- 172.21.15.80:0/1533379486 <== mon.1 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7a98004bd0 con 0x7f7aa8143820 2024-03-18T15:33:19.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.189+0000 7f7a9ffff640 1 -- 172.21.15.80:0/1533379486 <== mon.1 v2:172.21.15.80:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f7a9801b780 con 0x7f7aa8143820 2024-03-18T15:33:19.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.189+0000 7f7aae0d8640 1 -- 172.21.15.80:0/1533379486 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7aa804f550 con 0x7f7aa8143820 2024-03-18T15:33:19.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.193+0000 7f7a9ffff640 1 -- 172.21.15.80:0/1533379486 <== mon.1 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f7a98021070 con 0x7f7aa8143820 2024-03-18T15:33:19.262 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.261+0000 7f7aae0d8640 1 -- 172.21.15.80:0/1533379486 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c3e8bb76-e382-439d-b380-62ef92f1255a", "id": 2} v 0) -- 0x7f7aa806c960 con 0x7f7aa8143820 2024-03-18T15:33:19.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.265+0000 7f7a9ffff640 1 -- 172.21.15.80:0/1533379486 <== mon.1 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c3e8bb76-e382-439d-b380-62ef92f1255a", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f7a9801b360 con 0x7f7aa8143820 2024-03-18T15:33:19.267 INFO:teuthology.orchestra.run.smithi080.stdout:2 2024-03-18T15:33:19.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.265+0000 7f7aae0d8640 1 -- 172.21.15.80:0/1533379486 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7aa8143820 msgr2=0x7f7aa812fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:19.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.265+0000 7f7aae0d8640 1 --2- 172.21.15.80:0/1533379486 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7aa8143820 0x7f7aa812fec0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7a9800c4b0 tx=0x7f7a9800c970 comp rx=0 tx=0).stop 2024-03-18T15:33:19.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.265+0000 7f7aae0d8640 1 -- 172.21.15.80:0/1533379486 shutdown_connections 2024-03-18T15:33:19.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.265+0000 7f7aae0d8640 1 --2- 172.21.15.80:0/1533379486 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7aa8143820 0x7f7aa812fec0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:19.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.265+0000 7f7aae0d8640 1 --2- 172.21.15.80:0/1533379486 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7aa813dac0 0x7f7aa81e2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:19.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.265+0000 7f7aae0d8640 1 --2- 172.21.15.80:0/1533379486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7aa80afd40 0x7f7aa80ae2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:19.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.265+0000 7f7aae0d8640 1 -- 172.21.15.80:0/1533379486 >> 172.21.15.80:0/1533379486 conn(0x7f7aa80b9190 msgr2=0x7f7aa8142100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:19.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.265+0000 7f7aae0d8640 1 -- 172.21.15.80:0/1533379486 shutdown_connections 2024-03-18T15:33:19.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.265+0000 7f7aae0d8640 1 -- 172.21.15.80:0/1533379486 wait complete. 2024-03-18T15:33:19.283 DEBUG:teuthology.orchestra.run.smithi080:> sudo ceph --cluster ceph osd new e8c94625-f2cd-4eba-b53d-2e7356e1c02a 3 2024-03-18T15:33:19.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.393+0000 7feb66261640 1 Processor -- start 2024-03-18T15:33:19.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.393+0000 7feb66261640 1 -- start start 2024-03-18T15:33:19.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.393+0000 7feb66261640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7feb601449f0 0x7feb6014fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:19.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.393+0000 7feb66261640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb600b01e0 con 0x7feb600af4f0 2024-03-18T15:33:19.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.393+0000 7feb66261640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb600b9c20 con 0x7feb6013e450 2024-03-18T15:33:19.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.393+0000 7feb66261640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7feb600b99d0 con 0x7feb601449f0 2024-03-18T15:33:19.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.393+0000 7feb5f7fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7feb6013e450 0x7feb600afad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:50854/0 (socket says 172.21.15.80:50854) 2024-03-18T15:33:19.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.393+0000 7feb5f7fe640 1 -- 172.21.15.80:0/3091821751 learned_addr learned my addr 172.21.15.80:0/3091821751 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-03-18T15:33:19.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.393+0000 7feb5ffff640 1 --2- 172.21.15.80:0/3091821751 >> v2:172.21.15.5:3300/0 conn(0x7feb601449f0 0x7feb6014fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:19.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.393+0000 7feb5effd640 1 -- 172.21.15.80:0/3091821751 <== mon.1 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 120719068 0 0) 0x7feb600b9c20 con 0x7feb6013e450 2024-03-18T15:33:19.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.393+0000 7feb5effd640 1 -- 172.21.15.80:0/3091821751 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb4c003530 con 0x7feb6013e450 2024-03-18T15:33:19.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.393+0000 7feb5effd640 1 -- 172.21.15.80:0/3091821751 <== mon.1 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3275053681 0 0) 0x7feb4c003530 con 0x7feb6013e450 2024-03-18T15:33:19.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.393+0000 7feb5effd640 1 -- 172.21.15.80:0/3091821751 >> v2:172.21.15.5:3300/0 conn(0x7feb601449f0 msgr2=0x7feb6014fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:19.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.393+0000 7feb5effd640 1 --2- 172.21.15.80:0/3091821751 >> v2:172.21.15.5:3300/0 conn(0x7feb601449f0 0x7feb6014fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:19.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.393+0000 7feb5effd640 1 -- 172.21.15.80:0/3091821751 >> v1:172.21.15.5:6789/0 conn(0x7feb600af4f0 legacy=0x7feb601442e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:19.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.393+0000 7feb5effd640 1 -- 172.21.15.80:0/3091821751 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb600b9f00 con 0x7feb6013e450 2024-03-18T15:33:19.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.393+0000 7feb5effd640 1 -- 172.21.15.80:0/3091821751 <== mon.1 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3837549003 0 0) 0x7feb50001680 con 0x7feb6013e450 2024-03-18T15:33:19.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.393+0000 7feb5effd640 1 -- 172.21.15.80:0/3091821751 >> v1:172.21.15.80:6789/0 conn(0x7feb6013e450 legacy=0x7feb600afad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:19.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.393+0000 7feb5effd640 1 --2- 172.21.15.80:0/3091821751 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7feb4c003aa0 0x7feb4c003ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:19.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.393+0000 7feb5effd640 1 --2- 172.21.15.80:0/3091821751 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feb4c004810 0x7feb4c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:19.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.393+0000 7feb5effd640 1 --2- 172.21.15.80:0/3091821751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb4c0052f0 0x7feb4c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:19.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.393+0000 7feb5effd640 1 -- 172.21.15.80:0/3091821751 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7feb600b99d0 con 0x7feb4c0052f0 2024-03-18T15:33:19.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.393+0000 7feb5effd640 1 -- 172.21.15.80:0/3091821751 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7feb4c00c210 con 0x7feb4c004810 2024-03-18T15:33:19.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.393+0000 7feb5effd640 1 -- 172.21.15.80:0/3091821751 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7feb4c00c410 con 0x7feb4c003aa0 2024-03-18T15:33:19.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.393+0000 7feb57fff640 1 --2- 172.21.15.80:0/3091821751 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feb4c004810 0x7feb4c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:19.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.393+0000 7feb5f7fe640 1 --2- 172.21.15.80:0/3091821751 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7feb4c003aa0 0x7feb4c003ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:19.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.393+0000 7feb5ffff640 1 --2- 172.21.15.80:0/3091821751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb4c0052f0 0x7feb4c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:19.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.393+0000 7feb5f7fe640 1 -- 172.21.15.80:0/3091821751 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feb4c004810 msgr2=0x7feb4c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:19.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.393+0000 7feb5f7fe640 1 --2- 172.21.15.80:0/3091821751 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feb4c004810 0x7feb4c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:19.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.393+0000 7feb5f7fe640 1 -- 172.21.15.80:0/3091821751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb4c0052f0 msgr2=0x7feb4c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:19.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.393+0000 7feb5f7fe640 1 --2- 172.21.15.80:0/3091821751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb4c0052f0 0x7feb4c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:19.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.397+0000 7feb5f7fe640 1 -- 172.21.15.80:0/3091821751 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb4c003530 con 0x7feb4c003aa0 2024-03-18T15:33:19.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.397+0000 7feb5ffff640 1 -- 172.21.15.80:0/3091821751 reap_dead start 2024-03-18T15:33:19.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.397+0000 7feb57fff640 1 --2- 172.21.15.80:0/3091821751 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feb4c004810 0x7feb4c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:33:19.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.397+0000 7feb5f7fe640 1 --2- 172.21.15.80:0/3091821751 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7feb4c003aa0 0x7feb4c003ee0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7feb5000c1c0 tx=0x7feb5001f390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6df54fadd0b0cb55 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:19.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.397+0000 7feb5effd640 1 -- 172.21.15.80:0/3091821751 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb50007d00 con 0x7feb4c003aa0 2024-03-18T15:33:19.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.397+0000 7feb5effd640 1 -- 172.21.15.80:0/3091821751 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb50005b00 con 0x7feb4c003aa0 2024-03-18T15:33:19.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.397+0000 7feb5effd640 1 -- 172.21.15.80:0/3091821751 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb50007070 con 0x7feb4c003aa0 2024-03-18T15:33:19.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.397+0000 7feb66261640 1 -- 172.21.15.80:0/3091821751 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7feb4c003aa0 msgr2=0x7feb4c003ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:19.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.397+0000 7feb66261640 1 --2- 172.21.15.80:0/3091821751 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7feb4c003aa0 0x7feb4c003ee0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7feb5000c1c0 tx=0x7feb5001f390 comp rx=0 tx=0).stop 2024-03-18T15:33:19.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.397+0000 7feb66261640 1 -- 172.21.15.80:0/3091821751 shutdown_connections 2024-03-18T15:33:19.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.397+0000 7feb66261640 1 --2- 172.21.15.80:0/3091821751 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7feb4c003aa0 0x7feb4c003ee0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:19.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.397+0000 7feb66261640 1 -- 172.21.15.80:0/3091821751 >> 172.21.15.80:0/3091821751 conn(0x7feb6006e230 msgr2=0x7feb6006e640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:19.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.397+0000 7feb66261640 1 -- 172.21.15.80:0/3091821751 shutdown_connections 2024-03-18T15:33:19.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.397+0000 7feb66261640 1 -- 172.21.15.80:0/3091821751 wait complete. 2024-03-18T15:33:19.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.397+0000 7feb66261640 1 Processor -- start 2024-03-18T15:33:19.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.401+0000 7feb66261640 1 -- start start 2024-03-18T15:33:19.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.401+0000 7feb66261640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb4c003aa0 0x7feb601e05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:19.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.401+0000 7feb66261640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7feb601e0f50 0x7feb6013ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:19.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.401+0000 7feb66261640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feb60139d40 0x7feb6013a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:19.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.401+0000 7feb66261640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7feb600af050 con 0x7feb4c003aa0 2024-03-18T15:33:19.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.401+0000 7feb66261640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7feb600aeed0 con 0x7feb60139d40 2024-03-18T15:33:19.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.401+0000 7feb66261640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7feb600b5a30 con 0x7feb601e0f50 2024-03-18T15:33:19.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.401+0000 7feb57fff640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7feb601e0f50 0x7feb6013ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:19.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.401+0000 7feb5ffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feb60139d40 0x7feb6013a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:19.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.401+0000 7feb57fff640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7feb601e0f50 0x7feb6013ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.80:52374/0 (socket says 172.21.15.80:52374) 2024-03-18T15:33:19.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.401+0000 7feb5ffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feb60139d40 0x7feb6013a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50462/0 (socket says 172.21.15.80:50462) 2024-03-18T15:33:19.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.401+0000 7feb57fff640 1 -- 172.21.15.80:0/2291688957 learned_addr learned my addr 172.21.15.80:0/2291688957 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-03-18T15:33:19.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.401+0000 7feb5f7fe640 1 --2- 172.21.15.80:0/2291688957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb4c003aa0 0x7feb601e05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:19.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.401+0000 7feb57fff640 1 -- 172.21.15.80:0/2291688957 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feb60139d40 msgr2=0x7feb6013a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:19.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.401+0000 7feb57fff640 1 --2- 172.21.15.80:0/2291688957 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feb60139d40 0x7feb6013a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:19.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.401+0000 7feb57fff640 1 -- 172.21.15.80:0/2291688957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb4c003aa0 msgr2=0x7feb601e05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:19.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.401+0000 7feb57fff640 1 --2- 172.21.15.80:0/2291688957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb4c003aa0 0x7feb601e05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:19.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.401+0000 7feb57fff640 1 -- 172.21.15.80:0/2291688957 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb6013f440 con 0x7feb601e0f50 2024-03-18T15:33:19.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.401+0000 7feb57fff640 1 --2- 172.21.15.80:0/2291688957 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7feb601e0f50 0x7feb6013ef00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7feb4800f730 tx=0x7feb4800fbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:19.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.401+0000 7feb5cff9640 1 -- 172.21.15.80:0/2291688957 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb480013a0 con 0x7feb601e0f50 2024-03-18T15:33:19.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.401+0000 7feb5cff9640 1 -- 172.21.15.80:0/2291688957 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb48018070 con 0x7feb601e0f50 2024-03-18T15:33:19.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.401+0000 7feb5cff9640 1 -- 172.21.15.80:0/2291688957 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb480052e0 con 0x7feb601e0f50 2024-03-18T15:33:19.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.401+0000 7feb66261640 1 -- 172.21.15.80:0/2291688957 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb6013a700 con 0x7feb601e0f50 2024-03-18T15:33:19.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.401+0000 7feb66261640 1 -- 172.21.15.80:0/2291688957 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7feb6009aa60 con 0x7feb601e0f50 2024-03-18T15:33:19.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.401+0000 7feb5cff9640 1 -- 172.21.15.80:0/2291688957 <== mon.2 v2:172.21.15.80:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7feb4800d730 con 0x7feb601e0f50 2024-03-18T15:33:19.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.401+0000 7feb5cff9640 1 -- 172.21.15.80:0/2291688957 <== mon.2 v2:172.21.15.80:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7feb48003bf0 con 0x7feb601e0f50 2024-03-18T15:33:19.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.401+0000 7feb66261640 1 -- 172.21.15.80:0/2291688957 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb6004f550 con 0x7feb601e0f50 2024-03-18T15:33:19.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.405+0000 7feb5cff9640 1 -- 172.21.15.80:0/2291688957 <== mon.2 v2:172.21.15.80:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7feb48003600 con 0x7feb601e0f50 2024-03-18T15:33:19.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.469+0000 7feb66261640 1 -- 172.21.15.80:0/2291688957 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e8c94625-f2cd-4eba-b53d-2e7356e1c02a", "id": 3} v 0) -- 0x7feb6013d000 con 0x7feb601e0f50 2024-03-18T15:33:19.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.473+0000 7feb5cff9640 1 -- 172.21.15.80:0/2291688957 <== mon.2 v2:172.21.15.80:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e8c94625-f2cd-4eba-b53d-2e7356e1c02a", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7feb4800dbb0 con 0x7feb601e0f50 2024-03-18T15:33:19.475 INFO:teuthology.orchestra.run.smithi080.stdout:3 2024-03-18T15:33:19.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.473+0000 7feb66261640 1 -- 172.21.15.80:0/2291688957 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7feb601e0f50 msgr2=0x7feb6013ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:19.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.473+0000 7feb66261640 1 --2- 172.21.15.80:0/2291688957 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7feb601e0f50 0x7feb6013ef00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7feb4800f730 tx=0x7feb4800fbf0 comp rx=0 tx=0).stop 2024-03-18T15:33:19.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.473+0000 7feb66261640 1 -- 172.21.15.80:0/2291688957 shutdown_connections 2024-03-18T15:33:19.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.473+0000 7feb66261640 1 --2- 172.21.15.80:0/2291688957 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feb60139d40 0x7feb6013a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:19.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.473+0000 7feb66261640 1 --2- 172.21.15.80:0/2291688957 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7feb601e0f50 0x7feb6013ef00 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:19.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.473+0000 7feb66261640 1 --2- 172.21.15.80:0/2291688957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb4c003aa0 0x7feb601e05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:19.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.473+0000 7feb66261640 1 -- 172.21.15.80:0/2291688957 >> 172.21.15.80:0/2291688957 conn(0x7feb6006e230 msgr2=0x7feb6013b050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:19.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.473+0000 7feb66261640 1 -- 172.21.15.80:0/2291688957 shutdown_connections 2024-03-18T15:33:19.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.473+0000 7feb66261640 1 -- 172.21.15.80:0/2291688957 wait complete. 2024-03-18T15:33:19.491 DEBUG:teuthology.orchestra.run.smithi080:> sudo ceph --cluster ceph osd new 0099dd52-6265-4d32-82e4-3ef16e73add0 4 2024-03-18T15:33:19.582 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-03-18T15:33:19.581+0000 7fb3a3d19180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-18T15:33:19.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.597+0000 7f294e8c9640 1 Processor -- start 2024-03-18T15:33:19.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.597+0000 7f294e8c9640 1 -- start start 2024-03-18T15:33:19.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.601+0000 7f294e8c9640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f29480b6c60 0x7f2948148aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:19.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.601+0000 7f294e8c9640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f29480b5640 0x7f2948148fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:19.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.601+0000 7f294e8c9640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2948072b20 0x7f294806d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:19.603 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.601+0000 7f294e8c9640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f29480b6770 con 0x7f29480b6c60 2024-03-18T15:33:19.603 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.601+0000 7f294e8c9640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f29480b7770 con 0x7f29480b5640 2024-03-18T15:33:19.603 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.601+0000 7f294e8c9640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f29480b7a50 con 0x7f2948072b20 2024-03-18T15:33:19.603 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.601+0000 7f294ce66640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2948072b20 0x7f294806d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:19.603 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.601+0000 7f29477fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f29480b5640 0x7f2948148fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:19.603 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.601+0000 7f294ce66640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2948072b20 0x7f294806d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.80:52380/0 (socket says 172.21.15.80:52380) 2024-03-18T15:33:19.603 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.601+0000 7f294ce66640 1 -- 172.21.15.80:0/998562530 learned_addr learned my addr 172.21.15.80:0/998562530 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-03-18T15:33:19.603 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.601+0000 7f2947fff640 1 --2- 172.21.15.80:0/998562530 >> v2:172.21.15.5:3300/0 conn(0x7f29480b6c60 0x7f2948148aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:19.603 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.601+0000 7f294ce66640 1 -- 172.21.15.80:0/998562530 >> v2:172.21.15.80:3300/0 conn(0x7f29480b5640 msgr2=0x7f2948148fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:19.603 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.601+0000 7f294ce66640 1 --2- 172.21.15.80:0/998562530 >> v2:172.21.15.80:3300/0 conn(0x7f29480b5640 0x7f2948148fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:19.603 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.601+0000 7f294ce66640 1 -- 172.21.15.80:0/998562530 >> v2:172.21.15.5:3300/0 conn(0x7f29480b6c60 msgr2=0x7f2948148aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:19.603 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.601+0000 7f294ce66640 1 --2- 172.21.15.80:0/998562530 >> v2:172.21.15.5:3300/0 conn(0x7f29480b6c60 0x7f2948148aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:19.603 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.601+0000 7f294ce66640 1 -- 172.21.15.80:0/998562530 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29480b7040 con 0x7f2948072b20 2024-03-18T15:33:19.603 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.601+0000 7f29477fe640 1 --2- 172.21.15.80:0/998562530 >> v2:172.21.15.80:3300/0 conn(0x7f29480b5640 0x7f2948148fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:33:19.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.601+0000 7f2947fff640 1 --2- 172.21.15.80:0/998562530 >> v2:172.21.15.5:3300/0 conn(0x7f29480b6c60 0x7f2948148aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:33:19.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.601+0000 7f294ce66640 1 --2- 172.21.15.80:0/998562530 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2948072b20 0x7f294806d310 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f29480b76a0 tx=0x7f294001c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef9655b8af55f8b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:19.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.601+0000 7f2946ffd640 1 -- 172.21.15.80:0/998562530 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29400a5740 con 0x7f2948072b20 2024-03-18T15:33:19.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.601+0000 7f2946ffd640 1 -- 172.21.15.80:0/998562530 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29400a58e0 con 0x7f2948072b20 2024-03-18T15:33:19.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.601+0000 7f2946ffd640 1 -- 172.21.15.80:0/998562530 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29400a5cb0 con 0x7f2948072b20 2024-03-18T15:33:19.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.605+0000 7f294e8c9640 1 -- 172.21.15.80:0/998562530 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2948072b20 msgr2=0x7f294806d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:19.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.605+0000 7f294e8c9640 1 --2- 172.21.15.80:0/998562530 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2948072b20 0x7f294806d310 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f29480b76a0 tx=0x7f294001c8a0 comp rx=0 tx=0).stop 2024-03-18T15:33:19.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.605+0000 7f294e8c9640 1 -- 172.21.15.80:0/998562530 shutdown_connections 2024-03-18T15:33:19.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.605+0000 7f294e8c9640 1 --2- 172.21.15.80:0/998562530 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2948072b20 0x7f294806d310 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:19.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.605+0000 7f294e8c9640 1 --2- 172.21.15.80:0/998562530 >> v2:172.21.15.80:3300/0 conn(0x7f29480b5640 0x7f2948148fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:19.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.605+0000 7f294e8c9640 1 --2- 172.21.15.80:0/998562530 >> v2:172.21.15.5:3300/0 conn(0x7f29480b6c60 0x7f2948148aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:19.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.605+0000 7f294e8c9640 1 -- 172.21.15.80:0/998562530 >> 172.21.15.80:0/998562530 conn(0x7f2948066fc0 msgr2=0x7f29480673d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:19.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.605+0000 7f294e8c9640 1 -- 172.21.15.80:0/998562530 shutdown_connections 2024-03-18T15:33:19.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.605+0000 7f294e8c9640 1 -- 172.21.15.80:0/998562530 wait complete. 2024-03-18T15:33:19.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.605+0000 7f294e8c9640 1 Processor -- start 2024-03-18T15:33:19.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.605+0000 7f294e8c9640 1 -- start start 2024-03-18T15:33:19.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.605+0000 7f294e8c9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2948072b20 0x7f29481e7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:19.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.605+0000 7f294e8c9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29480b5640 0x7f294814a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:19.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.605+0000 7f294e8c9640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f29480b6c60 0x7f294814a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:19.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.605+0000 7f294e8c9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2948071fb0 con 0x7f29480b5640 2024-03-18T15:33:19.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.605+0000 7f294e8c9640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2948071e30 con 0x7f2948072b20 2024-03-18T15:33:19.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.605+0000 7f294e8c9640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f2948072130 con 0x7f29480b6c60 2024-03-18T15:33:19.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.605+0000 7f294ce66640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f29480b6c60 0x7f294814a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:19.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.605+0000 7f294ce66640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f29480b6c60 0x7f294814a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.80:52386/0 (socket says 172.21.15.80:52386) 2024-03-18T15:33:19.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.605+0000 7f294ce66640 1 -- 172.21.15.80:0/3080588753 learned_addr learned my addr 172.21.15.80:0/3080588753 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-03-18T15:33:19.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.605+0000 7f2947fff640 1 --2- 172.21.15.80:0/3080588753 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2948072b20 0x7f29481e7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:19.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.605+0000 7f29477fe640 1 --2- 172.21.15.80:0/3080588753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29480b5640 0x7f294814a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:19.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.605+0000 7f294ce66640 1 -- 172.21.15.80:0/3080588753 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2948072b20 msgr2=0x7f29481e7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:19.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.605+0000 7f294ce66640 1 --2- 172.21.15.80:0/3080588753 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2948072b20 0x7f29481e7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:19.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.605+0000 7f294ce66640 1 -- 172.21.15.80:0/3080588753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29480b5640 msgr2=0x7f294814a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:19.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.605+0000 7f294ce66640 1 --2- 172.21.15.80:0/3080588753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29480b5640 0x7f294814a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:19.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.605+0000 7f294ce66640 1 -- 172.21.15.80:0/3080588753 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29480b2570 con 0x7f29480b6c60 2024-03-18T15:33:19.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.605+0000 7f294ce66640 1 --2- 172.21.15.80:0/3080588753 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f29480b6c60 0x7f294814a5c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f29400af040 tx=0x7f2940019640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:19.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.605+0000 7f2944ff9640 1 -- 172.21.15.80:0/3080588753 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29400bc020 con 0x7f29480b6c60 2024-03-18T15:33:19.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.605+0000 7f2944ff9640 1 -- 172.21.15.80:0/3080588753 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29400b8670 con 0x7f29480b6c60 2024-03-18T15:33:19.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.605+0000 7f294e8c9640 1 -- 172.21.15.80:0/3080588753 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f294814ba80 con 0x7f29480b6c60 2024-03-18T15:33:19.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.605+0000 7f2944ff9640 1 -- 172.21.15.80:0/3080588753 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2940009510 con 0x7f29480b6c60 2024-03-18T15:33:19.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.605+0000 7f294e8c9640 1 -- 172.21.15.80:0/3080588753 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2948145ec0 con 0x7f29480b6c60 2024-03-18T15:33:19.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.605+0000 7f2944ff9640 1 -- 172.21.15.80:0/3080588753 <== mon.2 v2:172.21.15.80:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f29400bf020 con 0x7f29480b6c60 2024-03-18T15:33:19.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.605+0000 7f2944ff9640 1 -- 172.21.15.80:0/3080588753 <== mon.2 v2:172.21.15.80:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f29400b6440 con 0x7f29480b6c60 2024-03-18T15:33:19.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.605+0000 7f294e8c9640 1 -- 172.21.15.80:0/3080588753 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f294804f550 con 0x7f29480b6c60 2024-03-18T15:33:19.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.609+0000 7f2944ff9640 1 -- 172.21.15.80:0/3080588753 <== mon.2 v2:172.21.15.80:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f29400c6030 con 0x7f29480b6c60 2024-03-18T15:33:19.639 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-03-18T15:33:19.637+0000 7fb3a3d19180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-18T15:33:19.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.673+0000 7f294e8c9640 1 -- 172.21.15.80:0/3080588753 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_command({"prefix": "osd new", "uuid": "0099dd52-6265-4d32-82e4-3ef16e73add0", "id": 4} v 0) -- 0x7f29480b93d0 con 0x7f29480b6c60 2024-03-18T15:33:19.679 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-03-18T15:33:19.677+0000 7f43fab64180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-18T15:33:19.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.681+0000 7f2944ff9640 1 -- 172.21.15.80:0/3080588753 <== mon.2 v2:172.21.15.80:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0099dd52-6265-4d32-82e4-3ef16e73add0", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f29400165e0 con 0x7f29480b6c60 2024-03-18T15:33:19.683 INFO:teuthology.orchestra.run.smithi080.stdout:4 2024-03-18T15:33:19.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.681+0000 7f294e8c9640 1 -- 172.21.15.80:0/3080588753 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f29480b6c60 msgr2=0x7f294814a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:19.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.681+0000 7f294e8c9640 1 --2- 172.21.15.80:0/3080588753 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f29480b6c60 0x7f294814a5c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f29400af040 tx=0x7f2940019640 comp rx=0 tx=0).stop 2024-03-18T15:33:19.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.681+0000 7f294e8c9640 1 -- 172.21.15.80:0/3080588753 shutdown_connections 2024-03-18T15:33:19.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.681+0000 7f294e8c9640 1 --2- 172.21.15.80:0/3080588753 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f29480b6c60 0x7f294814a5c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:19.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.681+0000 7f294e8c9640 1 --2- 172.21.15.80:0/3080588753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29480b5640 0x7f294814a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:19.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.681+0000 7f294e8c9640 1 --2- 172.21.15.80:0/3080588753 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2948072b20 0x7f29481e7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:19.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.681+0000 7f294e8c9640 1 -- 172.21.15.80:0/3080588753 >> 172.21.15.80:0/3080588753 conn(0x7f2948066fc0 msgr2=0x7f2948142860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:19.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.681+0000 7f294e8c9640 1 -- 172.21.15.80:0/3080588753 shutdown_connections 2024-03-18T15:33:19.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.681+0000 7f294e8c9640 1 -- 172.21.15.80:0/3080588753 wait complete. 2024-03-18T15:33:19.694 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-03-18T15:33:19.693+0000 7fb3a3d19180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-18T15:33:19.699 DEBUG:teuthology.orchestra.run.smithi080:> sudo ceph --cluster ceph osd new ed10c84f-8c39-41cc-9b64-7c482a94a48e 5 2024-03-18T15:33:19.737 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-03-18T15:33:19.733+0000 7f43fab64180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-18T15:33:19.775 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-03-18T15:33:19.773+0000 7fb3a3d19180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-18T15:33:19.790 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-03-18T15:33:19.789+0000 7f43fab64180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-18T15:33:19.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.809+0000 7f641ff09640 1 Processor -- start 2024-03-18T15:33:19.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.809+0000 7f641ff09640 1 -- start start 2024-03-18T15:33:19.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.809+0000 7f641ff09640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f641806d070 0x7f64181489a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:19.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.809+0000 7f641ff09640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6418149370 con 0x7f64180b3b60 2024-03-18T15:33:19.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.809+0000 7f641ff09640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f641806e830 con 0x7f6418148ee0 2024-03-18T15:33:19.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.809+0000 7f641ff09640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f641806eb10 con 0x7f641806d070 2024-03-18T15:33:19.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.809+0000 7f641d4a4640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f641806d070 0x7f64181489a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:19.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.809+0000 7f641e4a6640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f6418148ee0 0x7f641814faf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:50868/0 (socket says 172.21.15.80:50868) 2024-03-18T15:33:19.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.809+0000 7f641e4a6640 1 -- 172.21.15.80:0/670273150 learned_addr learned my addr 172.21.15.80:0/670273150 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-03-18T15:33:19.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.809+0000 7f641d4a4640 1 -- 172.21.15.80:0/670273150 >> v1:172.21.15.80:6789/0 conn(0x7f6418148ee0 legacy=0x7f641814faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:19.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.809+0000 7f641d4a4640 1 -- 172.21.15.80:0/670273150 >> v1:172.21.15.5:6789/0 conn(0x7f64180b3b60 legacy=0x7f64180724c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:19.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.809+0000 7f641cca3640 1 -- 172.21.15.80:0/670273150 <== mon.1 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 104777053 0 0) 0x7f641806e830 con 0x7f6418148ee0 2024-03-18T15:33:19.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.813+0000 7f641d4a4640 1 -- 172.21.15.80:0/670273150 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f641806dd10 con 0x7f641806d070 2024-03-18T15:33:19.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.813+0000 7f641d4a4640 1 --2- 172.21.15.80:0/670273150 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f641806d070 0x7f64181489a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f640c0097c0 tx=0x7f640c01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=435205b2eea148bf server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:19.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.813+0000 7f641cca3640 1 -- 172.21.15.80:0/670273150 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640c0b0070 con 0x7f641806d070 2024-03-18T15:33:19.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.813+0000 7f641cca3640 1 -- 172.21.15.80:0/670273150 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f640c0ab600 con 0x7f641806d070 2024-03-18T15:33:19.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.813+0000 7f641cca3640 1 -- 172.21.15.80:0/670273150 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640c0ab9d0 con 0x7f641806d070 2024-03-18T15:33:19.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.813+0000 7f641ff09640 1 -- 172.21.15.80:0/670273150 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f641806d070 msgr2=0x7f64181489a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:19.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.813+0000 7f641ff09640 1 --2- 172.21.15.80:0/670273150 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f641806d070 0x7f64181489a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f640c0097c0 tx=0x7f640c01c410 comp rx=0 tx=0).stop 2024-03-18T15:33:19.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.813+0000 7f641ff09640 1 -- 172.21.15.80:0/670273150 shutdown_connections 2024-03-18T15:33:19.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.813+0000 7f641ff09640 1 --2- 172.21.15.80:0/670273150 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f641806d070 0x7f64181489a0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:19.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.813+0000 7f641ff09640 1 -- 172.21.15.80:0/670273150 >> 172.21.15.80:0/670273150 conn(0x7f64180b7a80 msgr2=0x7f64180b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:19.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.813+0000 7f641ff09640 1 -- 172.21.15.80:0/670273150 shutdown_connections 2024-03-18T15:33:19.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.813+0000 7f641ff09640 1 -- 172.21.15.80:0/670273150 wait complete. 2024-03-18T15:33:19.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.813+0000 7f641ff09640 1 Processor -- start 2024-03-18T15:33:19.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.817+0000 7f641ff09640 1 -- start start 2024-03-18T15:33:19.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.817+0000 7f641ff09640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f641806d070 0x7f64181459e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:19.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.817+0000 7f641ff09640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64180b3b60 0x7f641813e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:19.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.817+0000 7f641ff09640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6418148ee0 0x7f6418140ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:19.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.817+0000 7f641ff09640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6418153c40 con 0x7f64180b3b60 2024-03-18T15:33:19.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.817+0000 7f641ff09640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f641806e460 con 0x7f6418148ee0 2024-03-18T15:33:19.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.817+0000 7f641ff09640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f6418153dc0 con 0x7f641806d070 2024-03-18T15:33:19.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.817+0000 7f641e4a6640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6418148ee0 0x7f6418140ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:19.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.817+0000 7f641dca5640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f641806d070 0x7f64181459e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:19.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.817+0000 7f641dca5640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f641806d070 0x7f64181459e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.80:52394/0 (socket says 172.21.15.80:52394) 2024-03-18T15:33:19.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.817+0000 7f641e4a6640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6418148ee0 0x7f6418140ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50506/0 (socket says 172.21.15.80:50506) 2024-03-18T15:33:19.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.817+0000 7f641dca5640 1 -- 172.21.15.80:0/3525224652 learned_addr learned my addr 172.21.15.80:0/3525224652 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-03-18T15:33:19.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.817+0000 7f641d4a4640 1 --2- 172.21.15.80:0/3525224652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64180b3b60 0x7f641813e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:19.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.817+0000 7f641dca5640 1 -- 172.21.15.80:0/3525224652 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6418148ee0 msgr2=0x7f6418140ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:19.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.817+0000 7f641dca5640 1 --2- 172.21.15.80:0/3525224652 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6418148ee0 0x7f6418140ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:19.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.817+0000 7f641dca5640 1 -- 172.21.15.80:0/3525224652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64180b3b60 msgr2=0x7f641813e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:19.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.817+0000 7f641dca5640 1 --2- 172.21.15.80:0/3525224652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64180b3b60 0x7f641813e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:19.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.817+0000 7f641dca5640 1 -- 172.21.15.80:0/3525224652 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6418142020 con 0x7f641806d070 2024-03-18T15:33:19.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.817+0000 7f641e4a6640 1 --2- 172.21.15.80:0/3525224652 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6418148ee0 0x7f6418140ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:33:19.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.817+0000 7f641d4a4640 1 --2- 172.21.15.80:0/3525224652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64180b3b60 0x7f641813e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:33:19.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.817+0000 7f641dca5640 1 --2- 172.21.15.80:0/3525224652 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f641806d070 0x7f64181459e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f641400c530 tx=0x7f641400c9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:19.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.817+0000 7f640a7fc640 1 -- 172.21.15.80:0/3525224652 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6414016020 con 0x7f641806d070 2024-03-18T15:33:19.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.817+0000 7f640a7fc640 1 -- 172.21.15.80:0/3525224652 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6414005220 con 0x7f641806d070 2024-03-18T15:33:19.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.817+0000 7f641ff09640 1 -- 172.21.15.80:0/3525224652 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f641813c1e0 con 0x7f641806d070 2024-03-18T15:33:19.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.817+0000 7f640a7fc640 1 -- 172.21.15.80:0/3525224652 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6414007030 con 0x7f641806d070 2024-03-18T15:33:19.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.817+0000 7f641ff09640 1 -- 172.21.15.80:0/3525224652 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f641813d470 con 0x7f641806d070 2024-03-18T15:33:19.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.817+0000 7f640a7fc640 1 -- 172.21.15.80:0/3525224652 <== mon.2 v2:172.21.15.80:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6414007560 con 0x7f641806d070 2024-03-18T15:33:19.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.817+0000 7f640a7fc640 1 -- 172.21.15.80:0/3525224652 <== mon.2 v2:172.21.15.80:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f6414012c20 con 0x7f641806d070 2024-03-18T15:33:19.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.817+0000 7f641ff09640 1 -- 172.21.15.80:0/3525224652 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f641804f550 con 0x7f641806d070 2024-03-18T15:33:19.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.821+0000 7f640a7fc640 1 -- 172.21.15.80:0/3525224652 <== mon.2 v2:172.21.15.80:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f6414020020 con 0x7f641806d070 2024-03-18T15:33:19.830 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-03-18T15:33:19.829+0000 7fb3a3d19180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-18T15:33:19.871 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-03-18T15:33:19.869+0000 7f43fab64180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-18T15:33:19.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.889+0000 7f641ff09640 1 -- 172.21.15.80:0/3525224652 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ed10c84f-8c39-41cc-9b64-7c482a94a48e", "id": 5} v 0) -- 0x7f6418072e80 con 0x7f641806d070 2024-03-18T15:33:19.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.893+0000 7f640a7fc640 1 -- 172.21.15.80:0/3525224652 <== mon.2 v2:172.21.15.80:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ed10c84f-8c39-41cc-9b64-7c482a94a48e", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f6414012700 con 0x7f641806d070 2024-03-18T15:33:19.894 INFO:teuthology.orchestra.run.smithi080.stdout:5 2024-03-18T15:33:19.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.893+0000 7f641ff09640 1 -- 172.21.15.80:0/3525224652 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f641806d070 msgr2=0x7f64181459e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:19.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.893+0000 7f641ff09640 1 --2- 172.21.15.80:0/3525224652 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f641806d070 0x7f64181459e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f641400c530 tx=0x7f641400c9f0 comp rx=0 tx=0).stop 2024-03-18T15:33:19.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.893+0000 7f641ff09640 1 -- 172.21.15.80:0/3525224652 shutdown_connections 2024-03-18T15:33:19.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.893+0000 7f641ff09640 1 --2- 172.21.15.80:0/3525224652 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6418148ee0 0x7f6418140ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:19.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.893+0000 7f641ff09640 1 --2- 172.21.15.80:0/3525224652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f64180b3b60 0x7f641813e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:19.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.893+0000 7f641ff09640 1 --2- 172.21.15.80:0/3525224652 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f641806d070 0x7f64181459e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:19.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.893+0000 7f641ff09640 1 -- 172.21.15.80:0/3525224652 >> 172.21.15.80:0/3525224652 conn(0x7f64180b7a80 msgr2=0x7f64180b28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:19.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.893+0000 7f641ff09640 1 -- 172.21.15.80:0/3525224652 shutdown_connections 2024-03-18T15:33:19.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:19.893+0000 7f641ff09640 1 -- 172.21.15.80:0/3525224652 wait complete. 2024-03-18T15:33:19.911 DEBUG:teuthology.orchestra.run.smithi080:> sudo ceph --cluster ceph osd new c7a3538a-7048-4fba-8781-b440f03c25db 6 2024-03-18T15:33:19.911 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-03-18T15:33:19.909+0000 7fb3a3d19180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-18T15:33:19.925 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-03-18T15:33:19.925+0000 7f43fab64180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-18T15:33:19.968 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-03-18T15:33:19.965+0000 7fb3a3d19180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-18T15:33:20.007 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-03-18T15:33:20.005+0000 7f43fab64180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-18T15:33:20.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.037+0000 7f3c97178640 1 Processor -- start 2024-03-18T15:33:20.040 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.037+0000 7f3c97178640 1 -- start start 2024-03-18T15:33:20.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.037+0000 7f3c97178640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3c90072b20 0x7f3c900b68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:20.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.037+0000 7f3c97178640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3c900b6e30 0x7f3c90151d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:20.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.037+0000 7f3c97178640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c9006cc10 con 0x7f3c900b7930 2024-03-18T15:33:20.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.037+0000 7f3c97178640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f3c900b5e20 con 0x7f3c90072b20 2024-03-18T15:33:20.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.037+0000 7f3c97178640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f3c900b7d10 con 0x7f3c900b6e30 2024-03-18T15:33:20.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.037+0000 7f3c8ffff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3c90072b20 0x7f3c900b68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:20.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.037+0000 7f3c94f14640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f3c900b7930 0x7f3c90141430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:50872/0 (socket says 172.21.15.80:50872) 2024-03-18T15:33:20.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.037+0000 7f3c94f14640 1 -- 172.21.15.80:0/418648991 learned_addr learned my addr 172.21.15.80:0/418648991 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-03-18T15:33:20.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.037+0000 7f3c95715640 1 --2- 172.21.15.80:0/418648991 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3c900b6e30 0x7f3c90151d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:20.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.037+0000 7f3c8ffff640 1 -- 172.21.15.80:0/418648991 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3c900b6e30 msgr2=0x7f3c90151d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:20.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.037+0000 7f3c8ffff640 1 --2- 172.21.15.80:0/418648991 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3c900b6e30 0x7f3c90151d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:20.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.037+0000 7f3c8ffff640 1 -- 172.21.15.80:0/418648991 >> v1:172.21.15.80:6789/0 conn(0x7f3c900b7930 legacy=0x7f3c90141430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:20.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.037+0000 7f3c8ffff640 1 -- 172.21.15.80:0/418648991 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c900b6070 con 0x7f3c90072b20 2024-03-18T15:33:20.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.037+0000 7f3c8f7fe640 1 -- 172.21.15.80:0/418648991 <== mon.1 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2506999097 0 0) 0x7f3c9006cc10 con 0x7f3c900b7930 2024-03-18T15:33:20.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.037+0000 7f3c95715640 1 --2- 172.21.15.80:0/418648991 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3c900b6e30 0x7f3c90151d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:33:20.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.041+0000 7f3c8ffff640 1 --2- 172.21.15.80:0/418648991 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c90072b20 0x7f3c900b68f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3c840097c0 tx=0x7f3c8401c350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7b76fc7d93c1f64 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:20.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.041+0000 7f3c8f7fe640 1 -- 172.21.15.80:0/418648991 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c840b1070 con 0x7f3c90072b20 2024-03-18T15:33:20.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.041+0000 7f3c8f7fe640 1 -- 172.21.15.80:0/418648991 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c840ac640 con 0x7f3c90072b20 2024-03-18T15:33:20.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.041+0000 7f3c8f7fe640 1 -- 172.21.15.80:0/418648991 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c840aca10 con 0x7f3c90072b20 2024-03-18T15:33:20.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.041+0000 7f3c97178640 1 -- 172.21.15.80:0/418648991 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c90072b20 msgr2=0x7f3c900b68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:20.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.041+0000 7f3c97178640 1 --2- 172.21.15.80:0/418648991 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c90072b20 0x7f3c900b68f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3c840097c0 tx=0x7f3c8401c350 comp rx=0 tx=0).stop 2024-03-18T15:33:20.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.041+0000 7f3c97178640 1 -- 172.21.15.80:0/418648991 shutdown_connections 2024-03-18T15:33:20.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.041+0000 7f3c97178640 1 --2- 172.21.15.80:0/418648991 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3c900b6e30 0x7f3c90151d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:20.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.041+0000 7f3c97178640 1 --2- 172.21.15.80:0/418648991 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c90072b20 0x7f3c900b68f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:20.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.041+0000 7f3c97178640 1 -- 172.21.15.80:0/418648991 >> 172.21.15.80:0/418648991 conn(0x7f3c90066fc0 msgr2=0x7f3c900673d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:20.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.041+0000 7f3c97178640 1 -- 172.21.15.80:0/418648991 shutdown_connections 2024-03-18T15:33:20.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.041+0000 7f3c97178640 1 -- 172.21.15.80:0/418648991 wait complete. 2024-03-18T15:33:20.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.041+0000 7f3c97178640 1 Processor -- start 2024-03-18T15:33:20.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.045+0000 7f3c97178640 1 -- start start 2024-03-18T15:33:20.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.045+0000 7f3c97178640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3c90072b20 0x7f3c90134120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:20.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.045+0000 7f3c97178640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c900b6e30 0x7f3c9013ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:20.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.045+0000 7f3c97178640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c900b7930 0x7f3c901e2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:20.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.045+0000 7f3c97178640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3c900b58d0 con 0x7f3c900b6e30 2024-03-18T15:33:20.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.045+0000 7f3c97178640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9006d310 con 0x7f3c900b7930 2024-03-18T15:33:20.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.045+0000 7f3c97178640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f3c900b5a50 con 0x7f3c90072b20 2024-03-18T15:33:20.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.045+0000 7f3c95715640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c900b7930 0x7f3c901e2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:20.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.045+0000 7f3c94f14640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3c90072b20 0x7f3c90134120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:20.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.045+0000 7f3c95715640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c900b7930 0x7f3c901e2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50526/0 (socket says 172.21.15.80:50526) 2024-03-18T15:33:20.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.045+0000 7f3c95715640 1 -- 172.21.15.80:0/2571716119 learned_addr learned my addr 172.21.15.80:0/2571716119 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-03-18T15:33:20.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.045+0000 7f3c8ffff640 1 --2- 172.21.15.80:0/2571716119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c900b6e30 0x7f3c9013ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:20.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.045+0000 7f3c94f14640 1 -- 172.21.15.80:0/2571716119 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c900b7930 msgr2=0x7f3c901e2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:20.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.045+0000 7f3c94f14640 1 --2- 172.21.15.80:0/2571716119 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c900b7930 0x7f3c901e2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:20.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.045+0000 7f3c94f14640 1 -- 172.21.15.80:0/2571716119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c900b6e30 msgr2=0x7f3c9013ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:20.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.045+0000 7f3c94f14640 1 --2- 172.21.15.80:0/2571716119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c900b6e30 0x7f3c9013ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:20.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.045+0000 7f3c94f14640 1 -- 172.21.15.80:0/2571716119 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c900b9cf0 con 0x7f3c90072b20 2024-03-18T15:33:20.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.045+0000 7f3c95715640 1 --2- 172.21.15.80:0/2571716119 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c900b7930 0x7f3c901e2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:33:20.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.045+0000 7f3c8ffff640 1 --2- 172.21.15.80:0/2571716119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c900b6e30 0x7f3c9013ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:33:20.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.045+0000 7f3c94f14640 1 --2- 172.21.15.80:0/2571716119 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3c90072b20 0x7f3c90134120 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3c8000c440 tx=0x7f3c8000c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:20.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.045+0000 7f3c8d7fa640 1 -- 172.21.15.80:0/2571716119 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c800074e0 con 0x7f3c90072b20 2024-03-18T15:33:20.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.045+0000 7f3c8d7fa640 1 -- 172.21.15.80:0/2571716119 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c800047f0 con 0x7f3c90072b20 2024-03-18T15:33:20.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.045+0000 7f3c97178640 1 -- 172.21.15.80:0/2571716119 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c90135960 con 0x7f3c90072b20 2024-03-18T15:33:20.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.045+0000 7f3c8d7fa640 1 -- 172.21.15.80:0/2571716119 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c80004be0 con 0x7f3c90072b20 2024-03-18T15:33:20.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.045+0000 7f3c97178640 1 -- 172.21.15.80:0/2571716119 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c900ba870 con 0x7f3c90072b20 2024-03-18T15:33:20.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.045+0000 7f3c8d7fa640 1 -- 172.21.15.80:0/2571716119 <== mon.2 v2:172.21.15.80:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3c80011070 con 0x7f3c90072b20 2024-03-18T15:33:20.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.045+0000 7f3c8d7fa640 1 -- 172.21.15.80:0/2571716119 <== mon.2 v2:172.21.15.80:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f3c800139b0 con 0x7f3c90072b20 2024-03-18T15:33:20.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.045+0000 7f3c97178640 1 -- 172.21.15.80:0/2571716119 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c9004f550 con 0x7f3c90072b20 2024-03-18T15:33:20.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.049+0000 7f3c8d7fa640 1 -- 172.21.15.80:0/2571716119 <== mon.2 v2:172.21.15.80:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f3c80013c60 con 0x7f3c90072b20 2024-03-18T15:33:20.063 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-03-18T15:33:20.061+0000 7f43fab64180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-18T15:33:20.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.113+0000 7f3c97178640 1 -- 172.21.15.80:0/2571716119 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c7a3538a-7048-4fba-8781-b440f03c25db", "id": 6} v 0) -- 0x7f3c9006b890 con 0x7f3c90072b20 2024-03-18T15:33:20.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.117+0000 7f3c8d7fa640 1 -- 172.21.15.80:0/2571716119 <== mon.2 v2:172.21.15.80:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c7a3538a-7048-4fba-8781-b440f03c25db", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f3c80004e20 con 0x7f3c90072b20 2024-03-18T15:33:20.119 INFO:teuthology.orchestra.run.smithi080.stdout:6 2024-03-18T15:33:20.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.117+0000 7f3c97178640 1 -- 172.21.15.80:0/2571716119 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3c90072b20 msgr2=0x7f3c90134120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:20.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.117+0000 7f3c97178640 1 --2- 172.21.15.80:0/2571716119 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3c90072b20 0x7f3c90134120 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3c8000c440 tx=0x7f3c8000c900 comp rx=0 tx=0).stop 2024-03-18T15:33:20.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.117+0000 7f3c97178640 1 -- 172.21.15.80:0/2571716119 shutdown_connections 2024-03-18T15:33:20.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.117+0000 7f3c97178640 1 --2- 172.21.15.80:0/2571716119 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c900b7930 0x7f3c901e2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:20.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.117+0000 7f3c97178640 1 --2- 172.21.15.80:0/2571716119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c900b6e30 0x7f3c9013ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:20.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.117+0000 7f3c97178640 1 --2- 172.21.15.80:0/2571716119 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3c90072b20 0x7f3c90134120 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:20.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.117+0000 7f3c97178640 1 -- 172.21.15.80:0/2571716119 >> 172.21.15.80:0/2571716119 conn(0x7f3c90066fc0 msgr2=0x7f3c9013c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:20.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.117+0000 7f3c97178640 1 -- 172.21.15.80:0/2571716119 shutdown_connections 2024-03-18T15:33:20.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.117+0000 7f3c97178640 1 -- 172.21.15.80:0/2571716119 wait complete. 2024-03-18T15:33:20.125 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-03-18T15:33:20.121+0000 7fb3a3d19180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-18T15:33:20.139 DEBUG:teuthology.orchestra.run.smithi080:> sudo ceph --cluster ceph osd new f9418104-3a97-4110-bc3a-1f8151d0f757 7 2024-03-18T15:33:20.190 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-03-18T15:33:20.189+0000 7fb3a3d19180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-18T15:33:20.219 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-03-18T15:33:20.217+0000 7f43fab64180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-18T15:33:20.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.245+0000 7fa721176640 1 Processor -- start 2024-03-18T15:33:20.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.245+0000 7fa721176640 1 -- start start 2024-03-18T15:33:20.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.245+0000 7fa721176640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa71c0744d0 0x7fa71c1455b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:20.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.245+0000 7fa721176640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa71c142b40 0x7fa71c145af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:20.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.245+0000 7fa721176640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa71c0b55f0 con 0x7fa71c0b3050 2024-03-18T15:33:20.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.245+0000 7fa721176640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fa71c06fa50 con 0x7fa71c0744d0 2024-03-18T15:33:20.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.245+0000 7fa721176640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fa71c070010 con 0x7fa71c142b40 2024-03-18T15:33:20.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.245+0000 7fa71b59e640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa71c142b40 0x7fa71c145af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:20.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.245+0000 7fa71b59e640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa71c142b40 0x7fa71c145af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.80:52426/0 (socket says 172.21.15.80:52426) 2024-03-18T15:33:20.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.245+0000 7fa71b59e640 1 -- 172.21.15.80:0/2671644588 learned_addr learned my addr 172.21.15.80:0/2671644588 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-03-18T15:33:20.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.249+0000 7fa71ad9d640 1 --2- 172.21.15.80:0/2671644588 >> v2:172.21.15.5:3300/0 conn(0x7fa71c0744d0 0x7fa71c1455b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:20.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.249+0000 7fa71b59e640 1 -- 172.21.15.80:0/2671644588 >> v2:172.21.15.5:3300/0 conn(0x7fa71c0744d0 msgr2=0x7fa71c1455b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:20.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.249+0000 7fa71b59e640 1 --2- 172.21.15.80:0/2671644588 >> v2:172.21.15.5:3300/0 conn(0x7fa71c0744d0 0x7fa71c1455b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:20.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.249+0000 7fa71b59e640 1 -- 172.21.15.80:0/2671644588 >> v1:172.21.15.5:6789/0 conn(0x7fa71c0b3050 legacy=0x7fa71c142430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:20.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.249+0000 7fa71b59e640 1 -- 172.21.15.80:0/2671644588 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa71c0748b0 con 0x7fa71c142b40 2024-03-18T15:33:20.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.249+0000 7fa71b59e640 1 --2- 172.21.15.80:0/2671644588 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa71c142b40 0x7fa71c145af0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa714009e50 tx=0x7fa71401c780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=421878d420e8b3ad server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:20.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.249+0000 7fa719d9b640 1 -- 172.21.15.80:0/2671644588 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7140a5600 con 0x7fa71c142b40 2024-03-18T15:33:20.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.249+0000 7fa719d9b640 1 -- 172.21.15.80:0/2671644588 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7140a57a0 con 0x7fa71c142b40 2024-03-18T15:33:20.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.249+0000 7fa719d9b640 1 -- 172.21.15.80:0/2671644588 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7140a5b70 con 0x7fa71c142b40 2024-03-18T15:33:20.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.249+0000 7fa721176640 1 -- 172.21.15.80:0/2671644588 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa71c142b40 msgr2=0x7fa71c145af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:20.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.249+0000 7fa721176640 1 --2- 172.21.15.80:0/2671644588 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa71c142b40 0x7fa71c145af0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa714009e50 tx=0x7fa71401c780 comp rx=0 tx=0).stop 2024-03-18T15:33:20.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.249+0000 7fa721176640 1 -- 172.21.15.80:0/2671644588 shutdown_connections 2024-03-18T15:33:20.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.249+0000 7fa721176640 1 --2- 172.21.15.80:0/2671644588 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa71c142b40 0x7fa71c145af0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:20.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.249+0000 7fa721176640 1 --2- 172.21.15.80:0/2671644588 >> v2:172.21.15.5:3300/0 conn(0x7fa71c0744d0 0x7fa71c1455b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:20.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.249+0000 7fa721176640 1 -- 172.21.15.80:0/2671644588 >> 172.21.15.80:0/2671644588 conn(0x7fa71c06bad0 msgr2=0x7fa71c0b9190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:20.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.249+0000 7fa721176640 1 -- 172.21.15.80:0/2671644588 shutdown_connections 2024-03-18T15:33:20.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.249+0000 7fa721176640 1 -- 172.21.15.80:0/2671644588 wait complete. 2024-03-18T15:33:20.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.253+0000 7fa721176640 1 Processor -- start 2024-03-18T15:33:20.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.253+0000 7fa721176640 1 -- start start 2024-03-18T15:33:20.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.253+0000 7fa721176640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa71c0744d0 0x7fa71c13d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:20.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.253+0000 7fa721176640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa71c0b3050 0x7fa71c1417a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:20.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.253+0000 7fa721176640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa71c142b40 0x7fa71c1dd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:20.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.253+0000 7fa721176640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa71c1525d0 con 0x7fa71c142b40 2024-03-18T15:33:20.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.253+0000 7fa721176640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa71c152450 con 0x7fa71c0744d0 2024-03-18T15:33:20.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.253+0000 7fa721176640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fa71c152750 con 0x7fa71c0b3050 2024-03-18T15:33:20.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.253+0000 7fa71a59c640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa71c0b3050 0x7fa71c1417a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:20.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.253+0000 7fa71a59c640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa71c0b3050 0x7fa71c1417a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.80:52432/0 (socket says 172.21.15.80:52432) 2024-03-18T15:33:20.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.253+0000 7fa71a59c640 1 -- 172.21.15.80:0/419073189 learned_addr learned my addr 172.21.15.80:0/419073189 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-03-18T15:33:20.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.253+0000 7fa71ad9d640 1 --2- 172.21.15.80:0/419073189 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa71c0744d0 0x7fa71c13d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:20.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.253+0000 7fa71b59e640 1 --2- 172.21.15.80:0/419073189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa71c142b40 0x7fa71c1dd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:20.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.253+0000 7fa71a59c640 1 -- 172.21.15.80:0/419073189 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa71c0744d0 msgr2=0x7fa71c13d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:20.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.253+0000 7fa71a59c640 1 --2- 172.21.15.80:0/419073189 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa71c0744d0 0x7fa71c13d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:20.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.253+0000 7fa71a59c640 1 -- 172.21.15.80:0/419073189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa71c142b40 msgr2=0x7fa71c1dd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:20.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.253+0000 7fa71a59c640 1 --2- 172.21.15.80:0/419073189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa71c142b40 0x7fa71c1dd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:20.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.253+0000 7fa71a59c640 1 -- 172.21.15.80:0/419073189 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa71c141ce0 con 0x7fa71c0b3050 2024-03-18T15:33:20.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.253+0000 7fa71a59c640 1 --2- 172.21.15.80:0/419073189 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa71c0b3050 0x7fa71c1417a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa71000c430 tx=0x7fa71000c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:20.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.253+0000 7fa7037fe640 1 -- 172.21.15.80:0/419073189 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa710011790 con 0x7fa71c0b3050 2024-03-18T15:33:20.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.253+0000 7fa7037fe640 1 -- 172.21.15.80:0/419073189 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa710017070 con 0x7fa71c0b3050 2024-03-18T15:33:20.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.253+0000 7fa721176640 1 -- 172.21.15.80:0/419073189 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa71c0b8730 con 0x7fa71c0b3050 2024-03-18T15:33:20.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.253+0000 7fa7037fe640 1 -- 172.21.15.80:0/419073189 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa710011bb0 con 0x7fa71c0b3050 2024-03-18T15:33:20.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.253+0000 7fa721176640 1 -- 172.21.15.80:0/419073189 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa71c13e500 con 0x7fa71c0b3050 2024-03-18T15:33:20.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.253+0000 7fa7037fe640 1 -- 172.21.15.80:0/419073189 <== mon.2 v2:172.21.15.80:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa7100127b0 con 0x7fa71c0b3050 2024-03-18T15:33:20.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.253+0000 7fa7037fe640 1 -- 172.21.15.80:0/419073189 <== mon.2 v2:172.21.15.80:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fa71001ba50 con 0x7fa71c0b3050 2024-03-18T15:33:20.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.253+0000 7fa721176640 1 -- 172.21.15.80:0/419073189 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa71c04f550 con 0x7fa71c0b3050 2024-03-18T15:33:20.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.257+0000 7fa7037fe640 1 -- 172.21.15.80:0/419073189 <== mon.2 v2:172.21.15.80:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fa71c141ce0 con 0x7fa71c0b3050 2024-03-18T15:33:20.283 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-03-18T15:33:20.281+0000 7f43fab64180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-18T15:33:20.321 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.321+0000 7fa721176640 1 -- 172.21.15.80:0/419073189 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f9418104-3a97-4110-bc3a-1f8151d0f757", "id": 7} v 0) -- 0x7fa71c072200 con 0x7fa71c0b3050 2024-03-18T15:33:20.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.329+0000 7fa7037fe640 1 -- 172.21.15.80:0/419073189 <== mon.2 v2:172.21.15.80:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f9418104-3a97-4110-bc3a-1f8151d0f757", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fa710011d70 con 0x7fa71c0b3050 2024-03-18T15:33:20.329 INFO:teuthology.orchestra.run.smithi080.stdout:7 2024-03-18T15:33:20.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.329+0000 7fa721176640 1 -- 172.21.15.80:0/419073189 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa71c0b3050 msgr2=0x7fa71c1417a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:20.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.329+0000 7fa721176640 1 --2- 172.21.15.80:0/419073189 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa71c0b3050 0x7fa71c1417a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa71000c430 tx=0x7fa71000c8f0 comp rx=0 tx=0).stop 2024-03-18T15:33:20.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.329+0000 7fa721176640 1 -- 172.21.15.80:0/419073189 shutdown_connections 2024-03-18T15:33:20.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.329+0000 7fa721176640 1 --2- 172.21.15.80:0/419073189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa71c142b40 0x7fa71c1dd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:20.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.329+0000 7fa721176640 1 --2- 172.21.15.80:0/419073189 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa71c0b3050 0x7fa71c1417a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:20.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.329+0000 7fa721176640 1 --2- 172.21.15.80:0/419073189 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa71c0744d0 0x7fa71c13d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:20.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.329+0000 7fa721176640 1 -- 172.21.15.80:0/419073189 >> 172.21.15.80:0/419073189 conn(0x7fa71c06bad0 msgr2=0x7fa71c0b3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:20.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.329+0000 7fa721176640 1 -- 172.21.15.80:0/419073189 shutdown_connections 2024-03-18T15:33:20.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-03-18T15:33:20.329+0000 7fa721176640 1 -- 172.21.15.80:0/419073189 wait complete. 2024-03-18T15:33:20.346 INFO:tasks.ceph.osd.0:Restarting daemon 2024-03-18T15:33:20.346 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-03-18T15:33:20.349 INFO:tasks.ceph.osd.0:Started 2024-03-18T15:33:20.349 INFO:tasks.ceph.osd.1:Restarting daemon 2024-03-18T15:33:20.349 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-03-18T15:33:20.352 INFO:tasks.ceph.osd.1:Started 2024-03-18T15:33:20.352 INFO:tasks.ceph.osd.2:Restarting daemon 2024-03-18T15:33:20.352 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-03-18T15:33:20.354 INFO:tasks.ceph.osd.2:Started 2024-03-18T15:33:20.354 INFO:tasks.ceph.osd.3:Restarting daemon 2024-03-18T15:33:20.354 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-03-18T15:33:20.357 INFO:tasks.ceph.osd.3:Started 2024-03-18T15:33:20.357 INFO:tasks.ceph.osd.4:Restarting daemon 2024-03-18T15:33:20.357 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-03-18T15:33:20.359 INFO:tasks.ceph.osd.4:Started 2024-03-18T15:33:20.359 INFO:tasks.ceph.osd.5:Restarting daemon 2024-03-18T15:33:20.359 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-03-18T15:33:20.361 INFO:tasks.ceph.osd.5:Started 2024-03-18T15:33:20.361 INFO:tasks.ceph.osd.6:Restarting daemon 2024-03-18T15:33:20.361 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-03-18T15:33:20.363 INFO:tasks.ceph.osd.6:Started 2024-03-18T15:33:20.363 INFO:tasks.ceph.osd.7:Restarting daemon 2024-03-18T15:33:20.363 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-03-18T15:33:20.365 INFO:tasks.ceph.osd.7:Started 2024-03-18T15:33:20.366 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:33:20.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.485+0000 7fd7be975640 1 Processor -- start 2024-03-18T15:33:20.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.485+0000 7fd7be975640 1 -- start start 2024-03-18T15:33:20.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.485+0000 7fd7be975640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd7b80b5710 0x7fd7b814a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:20.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.485+0000 7fd7be975640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd7b814ae60 0x7fd7b8140ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:20.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.485+0000 7fd7be975640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7b80b9d50 con 0x7fd7b8141430 2024-03-18T15:33:20.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.485+0000 7fd7be975640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fd7b80b4320 con 0x7fd7b80b5710 2024-03-18T15:33:20.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.485+0000 7fd7be975640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fd7b80b4570 con 0x7fd7b814ae60 2024-03-18T15:33:20.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.485+0000 7fd7b7fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd7b80b5710 0x7fd7b814a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:20.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.485+0000 7fd7bcf12640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd7b8141430 0x7fd7b80b7690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60166/0 (socket says 172.21.15.5:60166) 2024-03-18T15:33:20.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.485+0000 7fd7bcf12640 1 -- 172.21.15.5:0/1942358579 learned_addr learned my addr 172.21.15.5:0/1942358579 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:33:20.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.485+0000 7fd7b77fe640 1 --2- 172.21.15.5:0/1942358579 >> v2:172.21.15.80:3300/0 conn(0x7fd7b814ae60 0x7fd7b8140ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:20.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.485+0000 7fd7b7fff640 1 -- 172.21.15.5:0/1942358579 >> v2:172.21.15.80:3300/0 conn(0x7fd7b814ae60 msgr2=0x7fd7b8140ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:20.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.485+0000 7fd7b7fff640 1 --2- 172.21.15.5:0/1942358579 >> v2:172.21.15.80:3300/0 conn(0x7fd7b814ae60 0x7fd7b8140ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:20.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.485+0000 7fd7b7fff640 1 -- 172.21.15.5:0/1942358579 >> v1:172.21.15.5:6789/0 conn(0x7fd7b8141430 legacy=0x7fd7b80b7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:20.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.485+0000 7fd7b7fff640 1 -- 172.21.15.5:0/1942358579 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7b80b4c30 con 0x7fd7b80b5710 2024-03-18T15:33:20.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.489+0000 7fd7b7fff640 1 --2- 172.21.15.5:0/1942358579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7b80b5710 0x7fd7b814a920 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd7b80b88c0 tx=0x7fd7ac01c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2689d1e52a3f8606 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:20.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.489+0000 7fd7b6ffd640 1 -- 172.21.15.5:0/1942358579 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac0b0070 con 0x7fd7b80b5710 2024-03-18T15:33:20.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.489+0000 7fd7b6ffd640 1 -- 172.21.15.5:0/1942358579 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7ac0ab600 con 0x7fd7b80b5710 2024-03-18T15:33:20.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.489+0000 7fd7b6ffd640 1 -- 172.21.15.5:0/1942358579 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac0ab9d0 con 0x7fd7b80b5710 2024-03-18T15:33:20.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.489+0000 7fd7be975640 1 -- 172.21.15.5:0/1942358579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7b80b5710 msgr2=0x7fd7b814a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:20.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.489+0000 7fd7be975640 1 --2- 172.21.15.5:0/1942358579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7b80b5710 0x7fd7b814a920 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd7b80b88c0 tx=0x7fd7ac01c360 comp rx=0 tx=0).stop 2024-03-18T15:33:20.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.489+0000 7fd7be975640 1 -- 172.21.15.5:0/1942358579 shutdown_connections 2024-03-18T15:33:20.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.489+0000 7fd7be975640 1 --2- 172.21.15.5:0/1942358579 >> v2:172.21.15.80:3300/0 conn(0x7fd7b814ae60 0x7fd7b8140ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:20.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.489+0000 7fd7be975640 1 --2- 172.21.15.5:0/1942358579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7b80b5710 0x7fd7b814a920 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:20.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.489+0000 7fd7be975640 1 -- 172.21.15.5:0/1942358579 >> 172.21.15.5:0/1942358579 conn(0x7fd7b80673c0 msgr2=0x7fd7b80677d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:20.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.489+0000 7fd7be975640 1 -- 172.21.15.5:0/1942358579 shutdown_connections 2024-03-18T15:33:20.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.489+0000 7fd7be975640 1 -- 172.21.15.5:0/1942358579 wait complete. 2024-03-18T15:33:20.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.489+0000 7fd7be975640 1 Processor -- start 2024-03-18T15:33:20.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.489+0000 7fd7be975640 1 -- start start 2024-03-18T15:33:20.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.489+0000 7fd7be975640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7b80b5710 0x7fd7b8146550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:20.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.489+0000 7fd7be975640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd7b8141430 0x7fd7b8132cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:20.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.489+0000 7fd7be975640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7b814ae60 0x7fd7b81331f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:20.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.489+0000 7fd7be975640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b80b5260 con 0x7fd7b814ae60 2024-03-18T15:33:20.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.489+0000 7fd7be975640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b80b50e0 con 0x7fd7b80b5710 2024-03-18T15:33:20.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.489+0000 7fd7be975640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fd7b8141840 con 0x7fd7b8141430 2024-03-18T15:33:20.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.493+0000 7fd7b7fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7b80b5710 0x7fd7b8146550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:20.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.493+0000 7fd7b7fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7b80b5710 0x7fd7b8146550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:55874/0 (socket says 172.21.15.5:55874) 2024-03-18T15:33:20.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.493+0000 7fd7b7fff640 1 -- 172.21.15.5:0/842610861 learned_addr learned my addr 172.21.15.5:0/842610861 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:20.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.493+0000 7fd7b77fe640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd7b8141430 0x7fd7b8132cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:20.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.493+0000 7fd7bcf12640 1 --2- 172.21.15.5:0/842610861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7b814ae60 0x7fd7b81331f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:20.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.493+0000 7fd7b77fe640 1 -- 172.21.15.5:0/842610861 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7b80b5710 msgr2=0x7fd7b8146550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:20.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.493+0000 7fd7b77fe640 1 --2- 172.21.15.5:0/842610861 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7b80b5710 0x7fd7b8146550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:20.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.493+0000 7fd7b77fe640 1 -- 172.21.15.5:0/842610861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7b814ae60 msgr2=0x7fd7b81331f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:20.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.493+0000 7fd7b77fe640 1 --2- 172.21.15.5:0/842610861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7b814ae60 0x7fd7b81331f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:20.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.493+0000 7fd7b77fe640 1 -- 172.21.15.5:0/842610861 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7b814a0d0 con 0x7fd7b8141430 2024-03-18T15:33:20.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.493+0000 7fd7b7fff640 1 --2- 172.21.15.5:0/842610861 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7b80b5710 0x7fd7b8146550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:33:20.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.493+0000 7fd7bcf12640 1 --2- 172.21.15.5:0/842610861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7b814ae60 0x7fd7b81331f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:33:20.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.493+0000 7fd7b77fe640 1 --2- 172.21.15.5:0/842610861 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd7b8141430 0x7fd7b8132cb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd7a800d600 tx=0x7fd7a800dac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:20.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.493+0000 7fd7b4ff9640 1 -- 172.21.15.5:0/842610861 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a80125f0 con 0x7fd7b8141430 2024-03-18T15:33:20.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.493+0000 7fd7b4ff9640 1 -- 172.21.15.5:0/842610861 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7a8019070 con 0x7fd7b8141430 2024-03-18T15:33:20.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.493+0000 7fd7be975640 1 -- 172.21.15.5:0/842610861 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7b80bbaa0 con 0x7fd7b8141430 2024-03-18T15:33:20.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.493+0000 7fd7b4ff9640 1 -- 172.21.15.5:0/842610861 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a80129b0 con 0x7fd7b8141430 2024-03-18T15:33:20.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.493+0000 7fd7be975640 1 -- 172.21.15.5:0/842610861 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7b80bb4e0 con 0x7fd7b8141430 2024-03-18T15:33:20.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.493+0000 7fd7b4ff9640 1 -- 172.21.15.5:0/842610861 <== mon.2 v2:172.21.15.80:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd7a8012b50 con 0x7fd7b8141430 2024-03-18T15:33:20.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.493+0000 7fd7b4ff9640 1 -- 172.21.15.5:0/842610861 <== mon.2 v2:172.21.15.80:3301/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fd7a8014be0 con 0x7fd7b8141430 2024-03-18T15:33:20.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.497+0000 7fd79e7fc640 1 -- 172.21.15.5:0/842610861 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7b804f550 con 0x7fd7b8141430 2024-03-18T15:33:20.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.497+0000 7fd7b4ff9640 1 -- 172.21.15.5:0/842610861 <== mon.2 v2:172.21.15.80:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fd7a80144d0 con 0x7fd7b8141430 2024-03-18T15:33:20.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.609+0000 7fd79e7fc640 1 -- 172.21.15.5:0/842610861 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd7b8132a10 con 0x7fd7b8141430 2024-03-18T15:33:20.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.609+0000 7fd7b4ff9640 1 -- 172.21.15.5:0/842610861 <== mon.2 v2:172.21.15.80:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fd7a80146b0 con 0x7fd7b8141430 2024-03-18T15:33:20.611 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:33:20.611 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":10,"fsid":"ef54fe03-5aa7-4eb2-9b1e-66c9cc345b94","created":"2024-03-18T15:33:18.086614+0000","modified":"2024-03-18T15:33:20.323840+0000","last_up_change":"0.000000","last_in_change":"2024-03-18T15:33:20.323840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"df1c754b-af59-4d16-8919-fbb6386b3c48","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b9a08b5e-8af4-4144-bdba-d69ed1ac627a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c3e8bb76-e382-439d-b380-62ef92f1255a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e8c94625-f2cd-4eba-b53d-2e7356e1c02a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"0099dd52-6265-4d32-82e4-3ef16e73add0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ed10c84f-8c39-41cc-9b64-7c482a94a48e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c7a3538a-7048-4fba-8781-b440f03c25db","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"f9418104-3a97-4110-bc3a-1f8151d0f757","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:33:20.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.609+0000 7fd79e7fc640 1 -- 172.21.15.5:0/842610861 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd7b8141430 msgr2=0x7fd7b8132cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:20.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.609+0000 7fd79e7fc640 1 --2- 172.21.15.5:0/842610861 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd7b8141430 0x7fd7b8132cb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd7a800d600 tx=0x7fd7a800dac0 comp rx=0 tx=0).stop 2024-03-18T15:33:20.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.609+0000 7fd79e7fc640 1 -- 172.21.15.5:0/842610861 shutdown_connections 2024-03-18T15:33:20.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.609+0000 7fd79e7fc640 1 --2- 172.21.15.5:0/842610861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd7b814ae60 0x7fd7b81331f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:20.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.609+0000 7fd79e7fc640 1 --2- 172.21.15.5:0/842610861 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd7b8141430 0x7fd7b8132cb0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:20.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.609+0000 7fd79e7fc640 1 --2- 172.21.15.5:0/842610861 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7b80b5710 0x7fd7b8146550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:20.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.609+0000 7fd79e7fc640 1 -- 172.21.15.5:0/842610861 >> 172.21.15.5:0/842610861 conn(0x7fd7b80673c0 msgr2=0x7fd7b813c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:20.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.609+0000 7fd79e7fc640 1 -- 172.21.15.5:0/842610861 shutdown_connections 2024-03-18T15:33:20.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:20.609+0000 7fd79e7fc640 1 -- 172.21.15.5:0/842610861 wait complete. 2024-03-18T15:33:20.627 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-03-18T15:33:20.628 INFO:tasks.ceph:Waiting for OSDs to come up 2024-03-18T15:33:20.802 INFO:tasks.ceph.mgr.y.smithi005.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-03-18T15:33:20.802 INFO:tasks.ceph.mgr.y.smithi005.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-03-18T15:33:20.802 INFO:tasks.ceph.mgr.y.smithi005.stderr: from numpy import show_config as show_numpy_config 2024-03-18T15:33:20.878 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-03-18T15:33:20.877+0000 7fb3a3d19180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-18T15:33:20.955 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-03-18T15:33:20.953+0000 7f7a64c917c0 -1 Falling back to public interface 2024-03-18T15:33:20.955 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-03-18T15:33:20.953+0000 7f8bae40c7c0 -1 Falling back to public interface 2024-03-18T15:33:20.956 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-03-18T15:33:20.953+0000 7f7a63c51640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2024-03-18T15:33:20.970 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-03-18T15:33:20.969+0000 7feae27407c0 -1 Falling back to public interface 2024-03-18T15:33:20.979 INFO:tasks.ceph.osd.7.smithi080.stderr:2024-03-18T15:33:20.977+0000 7f6f817217c0 -1 Falling back to public interface 2024-03-18T15:33:20.979 INFO:tasks.ceph.osd.6.smithi080.stderr:2024-03-18T15:33:20.977+0000 7f22274207c0 -1 Falling back to public interface 2024-03-18T15:33:20.986 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-03-18T15:33:20.985+0000 7f18fa3a47c0 -1 Falling back to public interface 2024-03-18T15:33:20.990 INFO:tasks.ceph.osd.4.smithi080.stderr:2024-03-18T15:33:20.989+0000 7f0cb93067c0 -1 Falling back to public interface 2024-03-18T15:33:20.996 INFO:tasks.ceph.mgr.x.smithi080.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-03-18T15:33:20.996 INFO:tasks.ceph.mgr.x.smithi080.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-03-18T15:33:20.996 INFO:tasks.ceph.mgr.x.smithi080.stderr: from numpy import show_config as show_numpy_config 2024-03-18T15:33:21.014 INFO:tasks.ceph.osd.5.smithi080.stderr:2024-03-18T15:33:21.013+0000 7f90c17ae7c0 -1 Falling back to public interface 2024-03-18T15:33:21.040 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-03-18T15:33:21.037+0000 7f43fab64180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-18T15:33:21.071 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-03-18T15:33:21.069+0000 7fb3a3d19180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-18T15:33:21.194 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-03-18T15:33:21.193+0000 7f43fab64180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-18T15:33:21.260 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-03-18T15:33:21.257+0000 7fb3a3d19180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-18T15:33:21.333 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-03-18T15:33:21.329+0000 7fb3a3d19180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-18T15:33:21.378 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-03-18T15:33:21.377+0000 7f43fab64180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-18T15:33:21.434 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-18T15:33:21.447 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-03-18T15:33:21.445+0000 7f43fab64180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-18T15:33:21.484 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-03-18T15:33:21.481+0000 7fb3a3d19180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-18T15:33:21.553 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.553+0000 7f2703b4b640 1 Processor -- start 2024-03-18T15:33:21.554 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.553+0000 7f2703b4b640 1 -- start start 2024-03-18T15:33:21.556 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.553+0000 7f2703b4b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f26fc072720 0x7f26fc150210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:21.557 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.553+0000 7f2703b4b640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26fc06d6a0 con 0x7f26fc0722c0 2024-03-18T15:33:21.557 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.553+0000 7f2703b4b640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26fc0b5df0 con 0x7f26fc06d2c0 2024-03-18T15:33:21.557 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.553+0000 7f2703b4b640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f26fc072b90 con 0x7f26fc072720 2024-03-18T15:33:21.557 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.553+0000 7f27010e6640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f26fc0722c0 0x7f26fc14cae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60176/0 (socket says 172.21.15.5:60176) 2024-03-18T15:33:21.557 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.553+0000 7f27010e6640 1 -- 172.21.15.5:0/3784226288 learned_addr learned my addr 172.21.15.5:0/3784226288 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:33:21.557 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.553+0000 7f27020e8640 1 --2- 172.21.15.5:0/3784226288 >> v2:172.21.15.80:3300/0 conn(0x7f26fc072720 0x7f26fc150210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:21.557 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.553+0000 7f27008e5640 1 -- 172.21.15.5:0/3784226288 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1770449959 0 0) 0x7f26fc06d6a0 con 0x7f26fc0722c0 2024-03-18T15:33:21.557 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.553+0000 7f27008e5640 1 -- 172.21.15.5:0/3784226288 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f26e8003530 con 0x7f26fc0722c0 2024-03-18T15:33:21.557 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.553+0000 7f27008e5640 1 -- 172.21.15.5:0/3784226288 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 167344009 0 0) 0x7f26e8003530 con 0x7f26fc0722c0 2024-03-18T15:33:21.557 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.553+0000 7f27008e5640 1 -- 172.21.15.5:0/3784226288 >> v2:172.21.15.80:3300/0 conn(0x7f26fc072720 msgr2=0x7f26fc150210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:21.557 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.553+0000 7f27008e5640 1 --2- 172.21.15.5:0/3784226288 >> v2:172.21.15.80:3300/0 conn(0x7f26fc072720 0x7f26fc150210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:21.557 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.553+0000 7f27008e5640 1 -- 172.21.15.5:0/3784226288 >> v1:172.21.15.80:6789/0 conn(0x7f26fc06d2c0 legacy=0x7f26fc144980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:21.557 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.553+0000 7f27008e5640 1 -- 172.21.15.5:0/3784226288 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26fc06e5a0 con 0x7f26fc0722c0 2024-03-18T15:33:21.558 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.553+0000 7f27008e5640 1 -- 172.21.15.5:0/3784226288 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3837549003 0 0) 0x7f26f0001680 con 0x7f26fc0722c0 2024-03-18T15:33:21.558 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.553+0000 7f27008e5640 1 -- 172.21.15.5:0/3784226288 >> v1:172.21.15.5:6789/0 conn(0x7f26fc0722c0 legacy=0x7f26fc14cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:21.558 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.553+0000 7f27008e5640 1 --2- 172.21.15.5:0/3784226288 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f26e80039c0 0x7f26e8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:21.558 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.553+0000 7f27008e5640 1 --2- 172.21.15.5:0/3784226288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26e8004810 0x7f26e8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:21.558 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.553+0000 7f27008e5640 1 --2- 172.21.15.5:0/3784226288 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26e80052a0 0x7f26e800dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:21.558 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.553+0000 7f27008e5640 1 -- 172.21.15.5:0/3784226288 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc072b90 con 0x7f26e8004810 2024-03-18T15:33:21.558 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.553+0000 7f27008e5640 1 -- 172.21.15.5:0/3784226288 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f26e80099f0 con 0x7f26e80052a0 2024-03-18T15:33:21.558 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.553+0000 7f27008e5640 1 -- 172.21.15.5:0/3784226288 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f26e8009bf0 con 0x7f26e80039c0 2024-03-18T15:33:21.558 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.553+0000 7f27010e6640 1 --2- 172.21.15.5:0/3784226288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26e8004810 0x7f26e8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:21.558 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.553+0000 7f27020e8640 1 --2- 172.21.15.5:0/3784226288 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26e80052a0 0x7f26e800dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:21.558 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.553+0000 7f27018e7640 1 --2- 172.21.15.5:0/3784226288 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f26e80039c0 0x7f26e8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:21.558 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.553+0000 7f27010e6640 1 -- 172.21.15.5:0/3784226288 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f26e80039c0 msgr2=0x7f26e8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:21.558 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.553+0000 7f27010e6640 1 --2- 172.21.15.5:0/3784226288 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f26e80039c0 0x7f26e8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:21.559 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.553+0000 7f27010e6640 1 -- 172.21.15.5:0/3784226288 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26e80052a0 msgr2=0x7f26e800dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:21.559 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.553+0000 7f27010e6640 1 --2- 172.21.15.5:0/3784226288 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26e80052a0 0x7f26e800dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:21.559 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.553+0000 7f27010e6640 1 -- 172.21.15.5:0/3784226288 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e8003530 con 0x7f26e8004810 2024-03-18T15:33:21.559 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.553+0000 7f27020e8640 1 -- 172.21.15.5:0/3784226288 reap_dead start 2024-03-18T15:33:21.559 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.553+0000 7f27018e7640 1 --2- 172.21.15.5:0/3784226288 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f26e80039c0 0x7f26e8003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:33:21.566 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.557+0000 7f27010e6640 1 --2- 172.21.15.5:0/3784226288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26e8004810 0x7f26e8004c90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f26f000c210 tx=0x7f26f001cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c07117f67ff766a4 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:21.566 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.557+0000 7f27008e5640 1 -- 172.21.15.5:0/3784226288 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f0007d40 con 0x7f26e8004810 2024-03-18T15:33:21.566 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.557+0000 7f27008e5640 1 -- 172.21.15.5:0/3784226288 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26f00a9070 con 0x7f26e8004810 2024-03-18T15:33:21.566 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.557+0000 7f27008e5640 1 -- 172.21.15.5:0/3784226288 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f0005d80 con 0x7f26e8004810 2024-03-18T15:33:21.566 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.557+0000 7f2703b4b640 1 -- 172.21.15.5:0/3784226288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26e8004810 msgr2=0x7f26e8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:21.567 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.557+0000 7f2703b4b640 1 --2- 172.21.15.5:0/3784226288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26e8004810 0x7f26e8004c90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f26f000c210 tx=0x7f26f001cc60 comp rx=0 tx=0).stop 2024-03-18T15:33:21.567 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.557+0000 7f2703b4b640 1 -- 172.21.15.5:0/3784226288 shutdown_connections 2024-03-18T15:33:21.567 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.557+0000 7f2703b4b640 1 --2- 172.21.15.5:0/3784226288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26e8004810 0x7f26e8004c90 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:21.567 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.557+0000 7f2703b4b640 1 -- 172.21.15.5:0/3784226288 >> 172.21.15.5:0/3784226288 conn(0x7f26fc06b2b0 msgr2=0x7f26fc06b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:21.567 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.557+0000 7f2703b4b640 1 -- 172.21.15.5:0/3784226288 shutdown_connections 2024-03-18T15:33:21.567 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.557+0000 7f2703b4b640 1 -- 172.21.15.5:0/3784226288 wait complete. 2024-03-18T15:33:21.567 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.557+0000 7f2703b4b640 1 Processor -- start 2024-03-18T15:33:21.567 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.557+0000 7f2703b4b640 1 -- start start 2024-03-18T15:33:21.567 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.557+0000 7f2703b4b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26e8004810 0x7f26fc0b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:21.567 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.557+0000 7f2703b4b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26fc146640 0x7f26fc141f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:21.567 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.557+0000 7f2703b4b640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f26fc142460 0x7f26fc133b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:21.567 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.557+0000 7f2703b4b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc147b50 con 0x7f26e8004810 2024-03-18T15:33:21.567 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.557+0000 7f2703b4b640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f26fc1479d0 con 0x7f26fc146640 2024-03-18T15:33:21.568 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.557+0000 7f2703b4b640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f26fc147cd0 con 0x7f26fc142460 2024-03-18T15:33:21.568 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.557+0000 7f27018e7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26e8004810 0x7f26fc0b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:21.568 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.557+0000 7f27018e7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26e8004810 0x7f26fc0b8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54440/0 (socket says 172.21.15.5:54440) 2024-03-18T15:33:21.568 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.557+0000 7f27018e7640 1 -- 172.21.15.5:0/722281724 learned_addr learned my addr 172.21.15.5:0/722281724 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:21.568 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.557+0000 7f27010e6640 1 --2- 172.21.15.5:0/722281724 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26fc146640 0x7f26fc141f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:21.568 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.557+0000 7f27020e8640 1 --2- 172.21.15.5:0/722281724 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f26fc142460 0x7f26fc133b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:21.568 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.557+0000 7f27018e7640 1 -- 172.21.15.5:0/722281724 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f26fc142460 msgr2=0x7f26fc133b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:21.568 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.557+0000 7f27018e7640 1 --2- 172.21.15.5:0/722281724 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f26fc142460 0x7f26fc133b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:21.568 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.557+0000 7f27018e7640 1 -- 172.21.15.5:0/722281724 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26fc146640 msgr2=0x7f26fc141f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:21.568 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.557+0000 7f27018e7640 1 --2- 172.21.15.5:0/722281724 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26fc146640 0x7f26fc141f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:21.568 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.557+0000 7f27018e7640 1 -- 172.21.15.5:0/722281724 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26fc147070 con 0x7f26e8004810 2024-03-18T15:33:21.568 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.557+0000 7f27020e8640 1 --2- 172.21.15.5:0/722281724 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f26fc142460 0x7f26fc133b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:33:21.568 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.557+0000 7f27010e6640 1 --2- 172.21.15.5:0/722281724 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26fc146640 0x7f26fc141f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:33:21.568 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.557+0000 7f27018e7640 1 --2- 172.21.15.5:0/722281724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26e8004810 0x7f26fc0b8b60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f26f800d4a0 tx=0x7f26f800d960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:21.569 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.561+0000 7f26ee7fc640 1 -- 172.21.15.5:0/722281724 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f8019070 con 0x7f26e8004810 2024-03-18T15:33:21.569 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.561+0000 7f26ee7fc640 1 -- 172.21.15.5:0/722281724 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26f80054a0 con 0x7f26e8004810 2024-03-18T15:33:21.569 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.561+0000 7f26ee7fc640 1 -- 172.21.15.5:0/722281724 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f8007270 con 0x7f26e8004810 2024-03-18T15:33:21.569 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.561+0000 7f2703b4b640 1 -- 172.21.15.5:0/722281724 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26fc147310 con 0x7f26e8004810 2024-03-18T15:33:21.569 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.561+0000 7f2703b4b640 1 -- 172.21.15.5:0/722281724 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26fc142d60 con 0x7f26e8004810 2024-03-18T15:33:21.569 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.561+0000 7f26ee7fc640 1 -- 172.21.15.5:0/722281724 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f26f8007410 con 0x7f26e8004810 2024-03-18T15:33:21.569 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.561+0000 7f26ee7fc640 1 -- 172.21.15.5:0/722281724 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f26f8003620 con 0x7f26e8004810 2024-03-18T15:33:21.569 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.561+0000 7f26dffff640 1 -- 172.21.15.5:0/722281724 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26d80052a0 con 0x7f26e8004810 2024-03-18T15:33:21.577 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.577+0000 7f26ee7fc640 1 -- 172.21.15.5:0/722281724 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f26f8022020 con 0x7f26e8004810 2024-03-18T15:33:21.593 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-03-18T15:33:21.593+0000 7f43fab64180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-18T15:33:21.635 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.633+0000 7f26dffff640 1 -- 172.21.15.5:0/722281724 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f26d8005860 con 0x7f26e8004810 2024-03-18T15:33:21.636 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.633+0000 7f26ee7fc640 1 -- 172.21.15.5:0/722281724 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f26f8003bd0 con 0x7f26e8004810 2024-03-18T15:33:21.636 INFO:teuthology.misc.health.smithi005.stdout: 2024-03-18T15:33:21.636 INFO:teuthology.misc.health.smithi005.stdout:{"epoch":10,"fsid":"ef54fe03-5aa7-4eb2-9b1e-66c9cc345b94","created":"2024-03-18T15:33:18.086614+0000","modified":"2024-03-18T15:33:20.323840+0000","last_up_change":"0.000000","last_in_change":"2024-03-18T15:33:20.323840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"df1c754b-af59-4d16-8919-fbb6386b3c48","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"b9a08b5e-8af4-4144-bdba-d69ed1ac627a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c3e8bb76-e382-439d-b380-62ef92f1255a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e8c94625-f2cd-4eba-b53d-2e7356e1c02a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"0099dd52-6265-4d32-82e4-3ef16e73add0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ed10c84f-8c39-41cc-9b64-7c482a94a48e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c7a3538a-7048-4fba-8781-b440f03c25db","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"f9418104-3a97-4110-bc3a-1f8151d0f757","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:33:21.637 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.633+0000 7f26dffff640 1 -- 172.21.15.5:0/722281724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26e8004810 msgr2=0x7f26fc0b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:21.637 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.633+0000 7f26dffff640 1 --2- 172.21.15.5:0/722281724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26e8004810 0x7f26fc0b8b60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f26f800d4a0 tx=0x7f26f800d960 comp rx=0 tx=0).stop 2024-03-18T15:33:21.637 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.633+0000 7f26dffff640 1 -- 172.21.15.5:0/722281724 shutdown_connections 2024-03-18T15:33:21.638 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.633+0000 7f26dffff640 1 --2- 172.21.15.5:0/722281724 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f26fc142460 0x7f26fc133b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:21.638 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.633+0000 7f26dffff640 1 --2- 172.21.15.5:0/722281724 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26fc146640 0x7f26fc141f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:21.638 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.633+0000 7f26dffff640 1 --2- 172.21.15.5:0/722281724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26e8004810 0x7f26fc0b8b60 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:21.638 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.633+0000 7f26dffff640 1 -- 172.21.15.5:0/722281724 >> 172.21.15.5:0/722281724 conn(0x7f26fc06b2b0 msgr2=0x7f26fc0b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:21.638 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.633+0000 7f26dffff640 1 -- 172.21.15.5:0/722281724 shutdown_connections 2024-03-18T15:33:21.638 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:21.633+0000 7f26dffff640 1 -- 172.21.15.5:0/722281724 wait complete. 2024-03-18T15:33:21.652 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-18T15:33:22.133 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-03-18T15:33:22.129+0000 7fb3a3d19180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-18T15:33:22.209 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-03-18T15:33:22.209+0000 7f43fab64180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-18T15:33:22.212 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-03-18T15:33:22.209+0000 7fb3a3d19180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-18T15:33:22.285 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-03-18T15:33:22.281+0000 7f43fab64180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-18T15:33:22.404 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-03-18T15:33:22.401+0000 7fb3a3d19180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-18T15:33:23.267 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-03-18T15:33:22.481+0000 7fb3a3d19180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-18T15:33:23.267 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-03-18T15:33:22.785+0000 7fb3a3d19180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-18T15:33:23.267 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-03-18T15:33:22.853+0000 7fb3a3d19180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-18T15:33:23.267 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-03-18T15:33:22.461+0000 7f43fab64180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-18T15:33:23.267 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-03-18T15:33:22.533+0000 7f43fab64180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-18T15:33:23.268 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-03-18T15:33:22.821+0000 7f43fab64180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-18T15:33:23.268 INFO:tasks.ceph.mgr.x.smithi080.stderr:2024-03-18T15:33:22.885+0000 7f43fab64180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-18T15:33:23.889 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-03-18T15:33:23.889+0000 7fb34b161640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-18T15:33:23.896 INFO:tasks.ceph.mgr.y.smithi005.stderr:2024-03-18T15:33:23.893+0000 7fb34b161640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-18T15:33:24.620 INFO:tasks.ceph.osd.7.smithi080.stderr:2024-03-18T15:33:24.617+0000 7f6f817217c0 -1 osd.7 0 log_to_monitors true 2024-03-18T15:33:24.785 INFO:tasks.ceph.osd.4.smithi080.stderr:2024-03-18T15:33:24.785+0000 7f0cb93067c0 -1 osd.4 0 log_to_monitors true 2024-03-18T15:33:24.830 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-03-18T15:33:24.829+0000 7f18fa3a47c0 -1 osd.3 0 log_to_monitors true 2024-03-18T15:33:24.846 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-03-18T15:33:24.845+0000 7f8bae40c7c0 -1 osd.1 0 log_to_monitors true 2024-03-18T15:33:24.949 INFO:tasks.ceph.osd.6.smithi080.stderr:2024-03-18T15:33:24.949+0000 7f22274207c0 -1 osd.6 0 log_to_monitors true 2024-03-18T15:33:24.955 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-03-18T15:33:24.953+0000 7f7a64c917c0 -1 osd.0 0 log_to_monitors true 2024-03-18T15:33:24.955 INFO:tasks.ceph.osd.5.smithi080.stderr:2024-03-18T15:33:24.953+0000 7f90c17ae7c0 -1 osd.5 0 log_to_monitors true 2024-03-18T15:33:24.960 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-03-18T15:33:24.957+0000 7feae27407c0 -1 osd.2 0 log_to_monitors true 2024-03-18T15:33:27.026 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-03-18T15:33:27.025+0000 7f18f634d640 -1 osd.3 0 waiting for initial osdmap 2024-03-18T15:33:27.062 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-03-18T15:33:27.061+0000 7f8baa3b5640 -1 osd.1 0 waiting for initial osdmap 2024-03-18T15:33:27.072 INFO:tasks.ceph.osd.3.smithi005.stderr:2024-03-18T15:33:27.069+0000 7f18ee182640 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T15:33:27.108 INFO:tasks.ceph.osd.1.smithi005.stderr:2024-03-18T15:33:27.105+0000 7f8ba21ea640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T15:33:27.546 INFO:tasks.ceph.osd.4.smithi080.stderr:2024-03-18T15:33:27.545+0000 7f0cb52af640 -1 osd.4 0 waiting for initial osdmap 2024-03-18T15:33:27.609 INFO:tasks.ceph.osd.4.smithi080.stderr:2024-03-18T15:33:27.609+0000 7f0cad0e4640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T15:33:27.644 INFO:tasks.ceph.osd.7.smithi080.stderr:2024-03-18T15:33:27.641+0000 7f6f7dedc640 -1 osd.7 0 waiting for initial osdmap 2024-03-18T15:33:27.675 INFO:tasks.ceph.osd.7.smithi080.stderr:2024-03-18T15:33:27.673+0000 7f6f754ff640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T15:33:27.975 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-03-18T15:33:27.973+0000 7f7a60c3a640 -1 osd.0 0 waiting for initial osdmap 2024-03-18T15:33:27.975 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-03-18T15:33:27.973+0000 7feade6e9640 -1 osd.2 0 waiting for initial osdmap 2024-03-18T15:33:27.994 INFO:tasks.ceph.osd.0.smithi005.stderr:2024-03-18T15:33:27.993+0000 7f7a58a6f640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T15:33:27.994 INFO:tasks.ceph.osd.2.smithi005.stderr:2024-03-18T15:33:27.993+0000 7fead651e640 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T15:33:28.226 INFO:tasks.ceph.osd.5.smithi080.stderr:2024-03-18T15:33:28.225+0000 7f90bdf69640 -1 osd.5 0 waiting for initial osdmap 2024-03-18T15:33:28.254 INFO:tasks.ceph.osd.5.smithi080.stderr:2024-03-18T15:33:28.253+0000 7f90b558c640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T15:33:28.291 INFO:tasks.ceph.osd.6.smithi080.stderr:2024-03-18T15:33:28.289+0000 7f2223bdb640 -1 osd.6 0 waiting for initial osdmap 2024-03-18T15:33:28.319 INFO:tasks.ceph.osd.6.smithi080.stderr:2024-03-18T15:33:28.317+0000 7f221b1fe640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-18T15:33:28.460 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-18T15:33:28.585 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.581+0000 7f02f78a5640 1 Processor -- start 2024-03-18T15:33:28.585 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.585+0000 7f02f78a5640 1 -- start start 2024-03-18T15:33:28.587 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.585+0000 7f02f78a5640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f02f009f170 0x7f02f009b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:28.587 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.585+0000 7f02f78a5640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f02f009f5b0 0x7f02f0151c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:28.587 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.585+0000 7f02f78a5640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02f00bbd60 con 0x7f02f0141850 2024-03-18T15:33:28.587 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.585+0000 7f02f78a5640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f02f00bac70 con 0x7f02f009f5b0 2024-03-18T15:33:28.587 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.585+0000 7f02f78a5640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f02f00bbb30 con 0x7f02f009f170 2024-03-18T15:33:28.587 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.585+0000 7f02f5e42640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f02f009f5b0 0x7f02f0151c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:28.587 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.585+0000 7f02f5641640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f02f0141850 0x7f02f0073390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60188/0 (socket says 172.21.15.5:60188) 2024-03-18T15:33:28.587 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.585+0000 7f02f5e42640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f02f009f5b0 0x7f02f0151c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54540/0 (socket says 172.21.15.5:54540) 2024-03-18T15:33:28.587 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.585+0000 7f02f5641640 1 -- 172.21.15.5:0/2281116893 learned_addr learned my addr 172.21.15.5:0/2281116893 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:33:28.587 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.585+0000 7f02f4e40640 1 --2- 172.21.15.5:0/2281116893 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f02f009f170 0x7f02f009b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:28.587 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.585+0000 7f02f5e42640 1 -- 172.21.15.5:0/2281116893 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f02f009f170 msgr2=0x7f02f009b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:28.587 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.585+0000 7f02f5e42640 1 --2- 172.21.15.5:0/2281116893 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f02f009f170 0x7f02f009b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:28.588 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.585+0000 7f02f5e42640 1 -- 172.21.15.5:0/2281116893 >> v1:172.21.15.5:6789/0 conn(0x7f02f0141850 legacy=0x7f02f0073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:28.588 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.585+0000 7f02effff640 1 -- 172.21.15.5:0/2281116893 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4022884262 0 0) 0x7f02f00bbd60 con 0x7f02f0141850 2024-03-18T15:33:28.588 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.585+0000 7f02f5e42640 1 -- 172.21.15.5:0/2281116893 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02f00bc040 con 0x7f02f009f5b0 2024-03-18T15:33:28.589 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.589+0000 7f02f5e42640 1 --2- 172.21.15.5:0/2281116893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02f009f5b0 0x7f02f0151c60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f02e00097c0 tx=0x7f02e001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28b390afa6f24e47 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:28.589 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.589+0000 7f02effff640 1 -- 172.21.15.5:0/2281116893 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e00b1070 con 0x7f02f009f5b0 2024-03-18T15:33:28.589 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.589+0000 7f02effff640 1 -- 172.21.15.5:0/2281116893 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02e00ac410 con 0x7f02f009f5b0 2024-03-18T15:33:28.590 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.589+0000 7f02effff640 1 -- 172.21.15.5:0/2281116893 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e00ac7e0 con 0x7f02f009f5b0 2024-03-18T15:33:28.591 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.589+0000 7f02f78a5640 1 -- 172.21.15.5:0/2281116893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02f009f5b0 msgr2=0x7f02f0151c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:28.591 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.589+0000 7f02f78a5640 1 --2- 172.21.15.5:0/2281116893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02f009f5b0 0x7f02f0151c60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f02e00097c0 tx=0x7f02e001c4a0 comp rx=0 tx=0).stop 2024-03-18T15:33:28.591 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.589+0000 7f02f78a5640 1 -- 172.21.15.5:0/2281116893 shutdown_connections 2024-03-18T15:33:28.591 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.589+0000 7f02f78a5640 1 --2- 172.21.15.5:0/2281116893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02f009f5b0 0x7f02f0151c60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:28.591 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.589+0000 7f02f78a5640 1 --2- 172.21.15.5:0/2281116893 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f02f009f170 0x7f02f009b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:28.591 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.589+0000 7f02f78a5640 1 -- 172.21.15.5:0/2281116893 >> 172.21.15.5:0/2281116893 conn(0x7f02f0067540 msgr2=0x7f02f009ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:28.591 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.589+0000 7f02f78a5640 1 -- 172.21.15.5:0/2281116893 shutdown_connections 2024-03-18T15:33:28.592 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.589+0000 7f02f78a5640 1 -- 172.21.15.5:0/2281116893 wait complete. 2024-03-18T15:33:28.592 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.589+0000 7f02f78a5640 1 Processor -- start 2024-03-18T15:33:28.592 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.589+0000 7f02f78a5640 1 -- start start 2024-03-18T15:33:28.593 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.589+0000 7f02f78a5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02f009f170 0x7f02f009fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:28.593 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.589+0000 7f02f78a5640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f02f009f5b0 0x7f02f013d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:28.593 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.589+0000 7f02f78a5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02f0141850 0x7f02f01e3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:28.594 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.589+0000 7f02f78a5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f02f0154d50 con 0x7f02f0141850 2024-03-18T15:33:28.594 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.589+0000 7f02f78a5640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f02f0154bd0 con 0x7f02f009f170 2024-03-18T15:33:28.594 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.589+0000 7f02f78a5640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f02f0154ed0 con 0x7f02f009f5b0 2024-03-18T15:33:28.594 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.589+0000 7f02f5e42640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02f0141850 0x7f02f01e3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:28.594 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.589+0000 7f02f5e42640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02f0141850 0x7f02f01e3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54546/0 (socket says 172.21.15.5:54546) 2024-03-18T15:33:28.594 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.589+0000 7f02f5e42640 1 -- 172.21.15.5:0/3410503811 learned_addr learned my addr 172.21.15.5:0/3410503811 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:28.594 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.589+0000 7f02f5641640 1 --2- 172.21.15.5:0/3410503811 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02f009f170 0x7f02f009fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:28.594 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.589+0000 7f02f5e42640 1 -- 172.21.15.5:0/3410503811 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f02f009f5b0 msgr2=0x7f02f013d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:28.594 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.589+0000 7f02f5e42640 1 --2- 172.21.15.5:0/3410503811 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f02f009f5b0 0x7f02f013d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:28.594 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.589+0000 7f02f5e42640 1 -- 172.21.15.5:0/3410503811 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02f009f170 msgr2=0x7f02f009fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:28.594 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.589+0000 7f02f5e42640 1 --2- 172.21.15.5:0/3410503811 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02f009f170 0x7f02f009fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:28.594 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.589+0000 7f02f5e42640 1 -- 172.21.15.5:0/3410503811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02f013ddb0 con 0x7f02f0141850 2024-03-18T15:33:28.595 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.589+0000 7f02f5e42640 1 --2- 172.21.15.5:0/3410503811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02f0141850 0x7f02f01e3e00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f02e00097c0 tx=0x7f02e0019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:28.595 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.589+0000 7f02edffb640 1 -- 172.21.15.5:0/3410503811 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e00b1040 con 0x7f02f0141850 2024-03-18T15:33:28.595 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.589+0000 7f02edffb640 1 -- 172.21.15.5:0/3410503811 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02e00ba070 con 0x7f02f0141850 2024-03-18T15:33:28.595 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.589+0000 7f02f78a5640 1 -- 172.21.15.5:0/3410503811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02f013e090 con 0x7f02f0141850 2024-03-18T15:33:28.595 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.589+0000 7f02edffb640 1 -- 172.21.15.5:0/3410503811 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e00b5670 con 0x7f02f0141850 2024-03-18T15:33:28.595 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.589+0000 7f02f78a5640 1 -- 172.21.15.5:0/3410503811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02f013d990 con 0x7f02f0141850 2024-03-18T15:33:28.595 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.593+0000 7f02f78a5640 1 -- 172.21.15.5:0/3410503811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02f0075450 con 0x7f02f0141850 2024-03-18T15:33:28.596 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.593+0000 7f02edffb640 1 -- 172.21.15.5:0/3410503811 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 4) ==== 87027+0+0 (secure 0 0 0) 0x7f02e00b5810 con 0x7f02f0141850 2024-03-18T15:33:28.596 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.593+0000 7f02edffb640 1 --2- 172.21.15.5:0/3410503811 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f02dc067600 0x7f02dc069ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:28.596 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.593+0000 7f02edffb640 1 -- 172.21.15.5:0/3410503811 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 3911+0+0 (secure 0 0 0) 0x7f02e011ee30 con 0x7f02f0141850 2024-03-18T15:33:28.596 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.593+0000 7f02f5641640 1 --2- 172.21.15.5:0/3410503811 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f02dc067600 0x7f02dc069ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:28.596 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.593+0000 7f02f5641640 1 --2- 172.21.15.5:0/3410503811 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f02dc067600 0x7f02dc069ac0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f02e8000f70 tx=0x7f02e8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:28.599 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.597+0000 7f02edffb640 1 -- 172.21.15.5:0/3410503811 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f02e0016090 con 0x7f02f0141850 2024-03-18T15:33:28.761 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.757+0000 7f02f78a5640 1 -- 172.21.15.5:0/3410503811 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f02f00b3dd0 con 0x7f02f0141850 2024-03-18T15:33:28.762 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.761+0000 7f02edffb640 1 -- 172.21.15.5:0/3410503811 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+9829 (secure 0 0 0) 0x7f02e00f0320 con 0x7f02f0141850 2024-03-18T15:33:28.763 INFO:teuthology.misc.health.smithi005.stdout: 2024-03-18T15:33:28.763 INFO:teuthology.misc.health.smithi005.stdout:{"epoch":14,"fsid":"ef54fe03-5aa7-4eb2-9b1e-66c9cc345b94","created":"2024-03-18T15:33:18.086614+0000","modified":"2024-03-18T15:33:28.048928+0000","last_up_change":"2024-03-18T15:33:28.048928+0000","last_in_change":"2024-03-18T15:33:20.323840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"df1c754b-af59-4d16-8919-fbb6386b3c48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6804","nonce":976264754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6808","nonce":976264754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6815","nonce":976264754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6811","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6813","nonce":976264754}]},"public_addr":"172.21.15.5:6804/976264754","cluster_addr":"172.21.15.5:6808/976264754","heartbeat_back_addr":"172.21.15.5:6815/976264754","heartbeat_front_addr":"172.21.15.5:6813/976264754","state":["exists","up"]},{"osd":1,"uuid":"b9a08b5e-8af4-4144-bdba-d69ed1ac627a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6801","nonce":3901761126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6803","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6805","nonce":3901761126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6812","nonce":3901761126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6807","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6809","nonce":3901761126}]},"public_addr":"172.21.15.5:6801/3901761126","cluster_addr":"172.21.15.5:6805/3901761126","heartbeat_back_addr":"172.21.15.5:6812/3901761126","heartbeat_front_addr":"172.21.15.5:6809/3901761126","state":["exists","up"]},{"osd":2,"uuid":"c3e8bb76-e382-439d-b380-62ef92f1255a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6817","nonce":502968448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6819","nonce":502968448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6823","nonce":502968448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6821","nonce":502968448}]},"public_addr":"172.21.15.5:6817/502968448","cluster_addr":"172.21.15.5:6819/502968448","heartbeat_back_addr":"172.21.15.5:6823/502968448","heartbeat_front_addr":"172.21.15.5:6821/502968448","state":["exists","up"]},{"osd":3,"uuid":"e8c94625-f2cd-4eba-b53d-2e7356e1c02a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6825","nonce":2425932735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6827","nonce":2425932735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6831","nonce":2425932735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6829","nonce":2425932735}]},"public_addr":"172.21.15.5:6825/2425932735","cluster_addr":"172.21.15.5:6827/2425932735","heartbeat_back_addr":"172.21.15.5:6831/2425932735","heartbeat_front_addr":"172.21.15.5:6829/2425932735","state":["exists","up"]},{"osd":4,"uuid":"0099dd52-6265-4d32-82e4-3ef16e73add0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6817","nonce":976514285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6819","nonce":976514285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6823","nonce":976514285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6821","nonce":976514285}]},"public_addr":"172.21.15.80:6817/976514285","cluster_addr":"172.21.15.80:6819/976514285","heartbeat_back_addr":"172.21.15.80:6823/976514285","heartbeat_front_addr":"172.21.15.80:6821/976514285","state":["exists","up"]},{"osd":5,"uuid":"ed10c84f-8c39-41cc-9b64-7c482a94a48e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c7a3538a-7048-4fba-8781-b440f03c25db","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"f9418104-3a97-4110-bc3a-1f8151d0f757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6801","nonce":982831666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6804","nonce":982831666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6812","nonce":982831666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6808","nonce":982831666}]},"public_addr":"172.21.15.80:6801/982831666","cluster_addr":"172.21.15.80:6804/982831666","heartbeat_back_addr":"172.21.15.80:6812/982831666","heartbeat_front_addr":"172.21.15.80:6808/982831666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:33:28.765 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.761+0000 7f02f78a5640 1 -- 172.21.15.5:0/3410503811 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f02dc067600 msgr2=0x7f02dc069ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:28.765 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.761+0000 7f02f78a5640 1 --2- 172.21.15.5:0/3410503811 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f02dc067600 0x7f02dc069ac0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f02e8000f70 tx=0x7f02e8009040 comp rx=0 tx=0).stop 2024-03-18T15:33:28.765 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.761+0000 7f02f78a5640 1 -- 172.21.15.5:0/3410503811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02f0141850 msgr2=0x7f02f01e3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:28.765 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.761+0000 7f02f78a5640 1 --2- 172.21.15.5:0/3410503811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02f0141850 0x7f02f01e3e00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f02e00097c0 tx=0x7f02e0019240 comp rx=0 tx=0).stop 2024-03-18T15:33:28.765 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.761+0000 7f02f78a5640 1 -- 172.21.15.5:0/3410503811 shutdown_connections 2024-03-18T15:33:28.765 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.761+0000 7f02f78a5640 1 --2- 172.21.15.5:0/3410503811 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f02dc067600 0x7f02dc069ac0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:28.765 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.761+0000 7f02f78a5640 1 --2- 172.21.15.5:0/3410503811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02f0141850 0x7f02f01e3e00 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:28.765 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.761+0000 7f02f78a5640 1 --2- 172.21.15.5:0/3410503811 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f02f009f5b0 0x7f02f013d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:28.765 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.761+0000 7f02f78a5640 1 --2- 172.21.15.5:0/3410503811 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02f009f170 0x7f02f009fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:28.765 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.761+0000 7f02f78a5640 1 -- 172.21.15.5:0/3410503811 >> 172.21.15.5:0/3410503811 conn(0x7f02f0067540 msgr2=0x7f02f013f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:28.766 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.761+0000 7f02f78a5640 1 -- 172.21.15.5:0/3410503811 shutdown_connections 2024-03-18T15:33:28.766 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:28.761+0000 7f02f78a5640 1 -- 172.21.15.5:0/3410503811 wait complete. 2024-03-18T15:33:28.779 DEBUG:teuthology.misc:6 of 8 OSDs are up 2024-03-18T15:33:35.586 DEBUG:teuthology.orchestra.run.smithi005:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-18T15:33:35.713 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.709+0000 7fa52d2e7640 1 Processor -- start 2024-03-18T15:33:35.713 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.709+0000 7fa52d2e7640 1 -- start start 2024-03-18T15:33:35.714 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.713+0000 7fa52d2e7640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa52806d200 0x7fa5280af0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:35.715 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.713+0000 7fa52d2e7640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa5280aea60 0x7fa5280af5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:35.715 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.713+0000 7fa52d2e7640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5280b07b0 con 0x7fa528072a90 2024-03-18T15:33:35.715 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.713+0000 7fa52d2e7640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fa5280b4970 con 0x7fa5280aea60 2024-03-18T15:33:35.715 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.713+0000 7fa52d2e7640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fa5280b4f30 con 0x7fa52806d200 2024-03-18T15:33:35.715 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.713+0000 7fa5267fc640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa5280aea60 0x7fa5280af5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:35.715 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.713+0000 7fa5277fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa528072a90 0x7fa5280ad9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:39318/0 (socket says 172.21.15.5:39318) 2024-03-18T15:33:35.715 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.713+0000 7fa5277fe640 1 -- 172.21.15.5:0/341783253 learned_addr learned my addr 172.21.15.5:0/341783253 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:33:35.715 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.713+0000 7fa526ffd640 1 --2- 172.21.15.5:0/341783253 >> v2:172.21.15.80:3300/0 conn(0x7fa52806d200 0x7fa5280af0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:35.715 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.713+0000 7fa5267fc640 1 -- 172.21.15.5:0/341783253 >> v2:172.21.15.80:3300/0 conn(0x7fa52806d200 msgr2=0x7fa5280af0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:35.715 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.713+0000 7fa5267fc640 1 --2- 172.21.15.5:0/341783253 >> v2:172.21.15.80:3300/0 conn(0x7fa52806d200 0x7fa5280af0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:35.715 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.713+0000 7fa5267fc640 1 -- 172.21.15.5:0/341783253 >> v1:172.21.15.5:6789/0 conn(0x7fa528072a90 legacy=0x7fa5280ad9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:35.715 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.713+0000 7fa525ffb640 1 -- 172.21.15.5:0/341783253 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3897850478 0 0) 0x7fa5280b07b0 con 0x7fa528072a90 2024-03-18T15:33:35.715 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.713+0000 7fa5267fc640 1 -- 172.21.15.5:0/341783253 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa52806d5e0 con 0x7fa5280aea60 2024-03-18T15:33:35.717 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.713+0000 7fa5267fc640 1 --2- 172.21.15.5:0/341783253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5280aea60 0x7fa5280af5e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa51c0097c0 tx=0x7fa51c01c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=474b85814126ea04 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:35.717 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.713+0000 7fa525ffb640 1 -- 172.21.15.5:0/341783253 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c0b0070 con 0x7fa5280aea60 2024-03-18T15:33:35.717 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.713+0000 7fa525ffb640 1 -- 172.21.15.5:0/341783253 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa51c0ab400 con 0x7fa5280aea60 2024-03-18T15:33:35.717 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.713+0000 7fa525ffb640 1 -- 172.21.15.5:0/341783253 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c0ab7d0 con 0x7fa5280aea60 2024-03-18T15:33:35.718 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.717+0000 7fa52d2e7640 1 -- 172.21.15.5:0/341783253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5280aea60 msgr2=0x7fa5280af5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:35.718 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.717+0000 7fa52d2e7640 1 --2- 172.21.15.5:0/341783253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5280aea60 0x7fa5280af5e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa51c0097c0 tx=0x7fa51c01c340 comp rx=0 tx=0).stop 2024-03-18T15:33:35.718 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.717+0000 7fa52d2e7640 1 -- 172.21.15.5:0/341783253 shutdown_connections 2024-03-18T15:33:35.718 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.717+0000 7fa52d2e7640 1 --2- 172.21.15.5:0/341783253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5280aea60 0x7fa5280af5e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:35.718 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.717+0000 7fa52d2e7640 1 --2- 172.21.15.5:0/341783253 >> v2:172.21.15.80:3300/0 conn(0x7fa52806d200 0x7fa5280af0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:35.718 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.717+0000 7fa52d2e7640 1 -- 172.21.15.5:0/341783253 >> 172.21.15.5:0/341783253 conn(0x7fa5280b5530 msgr2=0x7fa5280b5940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:35.718 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.717+0000 7fa52d2e7640 1 -- 172.21.15.5:0/341783253 shutdown_connections 2024-03-18T15:33:35.718 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.717+0000 7fa52d2e7640 1 -- 172.21.15.5:0/341783253 wait complete. 2024-03-18T15:33:35.719 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.717+0000 7fa52d2e7640 1 Processor -- start 2024-03-18T15:33:35.719 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.717+0000 7fa52d2e7640 1 -- start start 2024-03-18T15:33:35.721 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.717+0000 7fa52d2e7640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa52806d200 0x7fa528071880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:35.721 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.717+0000 7fa52d2e7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa528072a90 0x7fa52814cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:35.721 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.717+0000 7fa52d2e7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5280aea60 0x7fa52814d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:35.721 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.717+0000 7fa52d2e7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa528151900 con 0x7fa528072a90 2024-03-18T15:33:35.721 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.717+0000 7fa52d2e7640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa528151780 con 0x7fa5280aea60 2024-03-18T15:33:35.721 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.717+0000 7fa52d2e7640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fa528151a80 con 0x7fa52806d200 2024-03-18T15:33:35.721 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.717+0000 7fa5267fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa528072a90 0x7fa52814cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:35.721 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.717+0000 7fa5267fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa528072a90 0x7fa52814cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35944/0 (socket says 172.21.15.5:35944) 2024-03-18T15:33:35.721 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.717+0000 7fa5267fc640 1 -- 172.21.15.5:0/913662051 learned_addr learned my addr 172.21.15.5:0/913662051 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:35.721 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.717+0000 7fa5277fe640 1 --2- 172.21.15.5:0/913662051 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5280aea60 0x7fa52814d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:35.721 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.717+0000 7fa526ffd640 1 --2- 172.21.15.5:0/913662051 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa52806d200 0x7fa528071880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:35.721 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.717+0000 7fa5267fc640 1 -- 172.21.15.5:0/913662051 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa52806d200 msgr2=0x7fa528071880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:35.722 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.717+0000 7fa5267fc640 1 --2- 172.21.15.5:0/913662051 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa52806d200 0x7fa528071880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:35.722 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.717+0000 7fa5267fc640 1 -- 172.21.15.5:0/913662051 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5280aea60 msgr2=0x7fa52814d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:35.722 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.717+0000 7fa5267fc640 1 --2- 172.21.15.5:0/913662051 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5280aea60 0x7fa52814d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:35.722 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.717+0000 7fa5267fc640 1 -- 172.21.15.5:0/913662051 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa528070070 con 0x7fa528072a90 2024-03-18T15:33:35.722 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.717+0000 7fa5267fc640 1 --2- 172.21.15.5:0/913662051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa528072a90 0x7fa52814cfd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa51c0097c0 tx=0x7fa51c01ab80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:35.722 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.717+0000 7fa50f7fe640 1 -- 172.21.15.5:0/913662051 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c0bc070 con 0x7fa528072a90 2024-03-18T15:33:35.722 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.717+0000 7fa50f7fe640 1 -- 172.21.15.5:0/913662051 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa51c0b1040 con 0x7fa528072a90 2024-03-18T15:33:35.722 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.717+0000 7fa50f7fe640 1 -- 172.21.15.5:0/913662051 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c0b0050 con 0x7fa528072a90 2024-03-18T15:33:35.722 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.717+0000 7fa52d2e7640 1 -- 172.21.15.5:0/913662051 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5280b83c0 con 0x7fa528072a90 2024-03-18T15:33:35.722 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.717+0000 7fa52d2e7640 1 -- 172.21.15.5:0/913662051 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa528070350 con 0x7fa528072a90 2024-03-18T15:33:35.723 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.721+0000 7fa52d2e7640 1 -- 172.21.15.5:0/913662051 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa528075450 con 0x7fa528072a90 2024-03-18T15:33:35.723 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.721+0000 7fa50f7fe640 1 -- 172.21.15.5:0/913662051 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fa51c0bf070 con 0x7fa528072a90 2024-03-18T15:33:35.723 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.721+0000 7fa50f7fe640 1 --2- 172.21.15.5:0/913662051 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa514067690 0x7fa514069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:35.723 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.721+0000 7fa50f7fe640 1 -- 172.21.15.5:0/913662051 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fa51c11f4f0 con 0x7fa528072a90 2024-03-18T15:33:35.723 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.721+0000 7fa526ffd640 1 --2- 172.21.15.5:0/913662051 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa514067690 0x7fa514069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:35.723 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.721+0000 7fa526ffd640 1 --2- 172.21.15.5:0/913662051 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa514067690 0x7fa514069b50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa52813fc00 tx=0x7fa518008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:35.727 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.725+0000 7fa50f7fe640 1 -- 172.21.15.5:0/913662051 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fa51c0c6020 con 0x7fa528072a90 2024-03-18T15:33:35.887 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.885+0000 7fa52d2e7640 1 -- 172.21.15.5:0/913662051 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa5280b1cd0 con 0x7fa528072a90 2024-03-18T15:33:35.889 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.889+0000 7fa50f7fe640 1 -- 172.21.15.5:0/913662051 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12728 (secure 0 0 0) 0x7fa528070070 con 0x7fa528072a90 2024-03-18T15:33:35.889 INFO:teuthology.misc.health.smithi005.stdout: 2024-03-18T15:33:35.890 INFO:teuthology.misc.health.smithi005.stdout:{"epoch":17,"fsid":"ef54fe03-5aa7-4eb2-9b1e-66c9cc345b94","created":"2024-03-18T15:33:18.086614+0000","modified":"2024-03-18T15:33:31.235982+0000","last_up_change":"2024-03-18T15:33:29.229911+0000","last_in_change":"2024-03-18T15:33:20.323840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:28.963186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df1c754b-af59-4d16-8919-fbb6386b3c48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6804","nonce":976264754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6808","nonce":976264754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6815","nonce":976264754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6811","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6813","nonce":976264754}]},"public_addr":"172.21.15.5:6804/976264754","cluster_addr":"172.21.15.5:6808/976264754","heartbeat_back_addr":"172.21.15.5:6815/976264754","heartbeat_front_addr":"172.21.15.5:6813/976264754","state":["exists","up"]},{"osd":1,"uuid":"b9a08b5e-8af4-4144-bdba-d69ed1ac627a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6801","nonce":3901761126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6803","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6805","nonce":3901761126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6812","nonce":3901761126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6807","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6809","nonce":3901761126}]},"public_addr":"172.21.15.5:6801/3901761126","cluster_addr":"172.21.15.5:6805/3901761126","heartbeat_back_addr":"172.21.15.5:6812/3901761126","heartbeat_front_addr":"172.21.15.5:6809/3901761126","state":["exists","up"]},{"osd":2,"uuid":"c3e8bb76-e382-439d-b380-62ef92f1255a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6817","nonce":502968448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6819","nonce":502968448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6823","nonce":502968448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6821","nonce":502968448}]},"public_addr":"172.21.15.5:6817/502968448","cluster_addr":"172.21.15.5:6819/502968448","heartbeat_back_addr":"172.21.15.5:6823/502968448","heartbeat_front_addr":"172.21.15.5:6821/502968448","state":["exists","up"]},{"osd":3,"uuid":"e8c94625-f2cd-4eba-b53d-2e7356e1c02a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6825","nonce":2425932735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6827","nonce":2425932735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6831","nonce":2425932735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6829","nonce":2425932735}]},"public_addr":"172.21.15.5:6825/2425932735","cluster_addr":"172.21.15.5:6827/2425932735","heartbeat_back_addr":"172.21.15.5:6831/2425932735","heartbeat_front_addr":"172.21.15.5:6829/2425932735","state":["exists","up"]},{"osd":4,"uuid":"0099dd52-6265-4d32-82e4-3ef16e73add0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6817","nonce":976514285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6819","nonce":976514285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6823","nonce":976514285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6821","nonce":976514285}]},"public_addr":"172.21.15.80:6817/976514285","cluster_addr":"172.21.15.80:6819/976514285","heartbeat_back_addr":"172.21.15.80:6823/976514285","heartbeat_front_addr":"172.21.15.80:6821/976514285","state":["exists","up"]},{"osd":5,"uuid":"ed10c84f-8c39-41cc-9b64-7c482a94a48e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6825","nonce":627153447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6827","nonce":627153447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6831","nonce":627153447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6829","nonce":627153447}]},"public_addr":"172.21.15.80:6825/627153447","cluster_addr":"172.21.15.80:6827/627153447","heartbeat_back_addr":"172.21.15.80:6831/627153447","heartbeat_front_addr":"172.21.15.80:6829/627153447","state":["exists","up"]},{"osd":6,"uuid":"c7a3538a-7048-4fba-8781-b440f03c25db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6803","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6805","nonce":488342527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6807","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6809","nonce":488342527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6815","nonce":488342527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6811","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6813","nonce":488342527}]},"public_addr":"172.21.15.80:6805/488342527","cluster_addr":"172.21.15.80:6809/488342527","heartbeat_back_addr":"172.21.15.80:6815/488342527","heartbeat_front_addr":"172.21.15.80:6813/488342527","state":["exists","up"]},{"osd":7,"uuid":"f9418104-3a97-4110-bc3a-1f8151d0f757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6801","nonce":982831666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6804","nonce":982831666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6812","nonce":982831666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6808","nonce":982831666}]},"public_addr":"172.21.15.80:6801/982831666","cluster_addr":"172.21.15.80:6804/982831666","heartbeat_back_addr":"172.21.15.80:6812/982831666","heartbeat_front_addr":"172.21.15.80:6808/982831666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.970828+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.858116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.987556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.864603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.756293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.591502+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:33:35.891 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.889+0000 7fa52d2e7640 1 -- 172.21.15.5:0/913662051 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa514067690 msgr2=0x7fa514069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:35.891 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.889+0000 7fa52d2e7640 1 --2- 172.21.15.5:0/913662051 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa514067690 0x7fa514069b50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa52813fc00 tx=0x7fa518008040 comp rx=0 tx=0).stop 2024-03-18T15:33:35.892 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.889+0000 7fa52d2e7640 1 -- 172.21.15.5:0/913662051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa528072a90 msgr2=0x7fa52814cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:35.892 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.889+0000 7fa52d2e7640 1 --2- 172.21.15.5:0/913662051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa528072a90 0x7fa52814cfd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa51c0097c0 tx=0x7fa51c01ab80 comp rx=0 tx=0).stop 2024-03-18T15:33:35.892 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.889+0000 7fa52d2e7640 1 -- 172.21.15.5:0/913662051 shutdown_connections 2024-03-18T15:33:35.892 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.889+0000 7fa52d2e7640 1 --2- 172.21.15.5:0/913662051 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa514067690 0x7fa514069b50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:35.892 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.889+0000 7fa52d2e7640 1 --2- 172.21.15.5:0/913662051 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5280aea60 0x7fa52814d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:35.892 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.889+0000 7fa52d2e7640 1 --2- 172.21.15.5:0/913662051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa528072a90 0x7fa52814cfd0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:35.892 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.889+0000 7fa52d2e7640 1 --2- 172.21.15.5:0/913662051 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa52806d200 0x7fa528071880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:35.892 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.889+0000 7fa52d2e7640 1 -- 172.21.15.5:0/913662051 >> 172.21.15.5:0/913662051 conn(0x7fa5280b5530 msgr2=0x7fa5280b1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:35.892 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.889+0000 7fa52d2e7640 1 -- 172.21.15.5:0/913662051 shutdown_connections 2024-03-18T15:33:35.892 INFO:teuthology.misc.health.smithi005.stderr:2024-03-18T15:33:35.889+0000 7fa52d2e7640 1 -- 172.21.15.5:0/913662051 wait complete. 2024-03-18T15:33:35.907 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-03-18T15:33:35.907 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-03-18T15:33:35.907 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-18T15:33:35.907 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-18T15:33:35.910 INFO:tasks.ceph.mds.a:Started 2024-03-18T15:33:35.910 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-18T15:33:35.910 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-18T15:33:35.912 INFO:tasks.ceph.mds.b:Started 2024-03-18T15:33:35.913 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-03-18T15:33:35.913 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:33:35.977 INFO:tasks.ceph.mds.a.smithi005.stdout:starting mds.a at 2024-03-18T15:33:35.997 INFO:tasks.ceph.mds.b.smithi080.stdout:starting mds.b at 2024-03-18T15:33:36.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.029+0000 7f8816aa4640 1 Processor -- start 2024-03-18T15:33:36.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.029+0000 7f8816aa4640 1 -- start start 2024-03-18T15:33:36.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.033+0000 7f8816aa4640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f881006d600 0x7f8810141380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:36.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.033+0000 7f8816aa4640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f881014b330 0x7f8810151d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:36.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.033+0000 7f8816aa4640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f881006e2e0 con 0x7f881006d9e0 2024-03-18T15:33:36.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.033+0000 7f8816aa4640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f88100b5bc0 con 0x7f881006d600 2024-03-18T15:33:36.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.033+0000 7f8816aa4640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f88100b5850 con 0x7f881014b330 2024-03-18T15:33:36.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.033+0000 7f8814840640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f881006d600 0x7f8810141380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:36.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.033+0000 7f880ffff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f881006d9e0 0x7f881014ac20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:39340/0 (socket says 172.21.15.5:39340) 2024-03-18T15:33:36.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.033+0000 7f8814840640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f881006d600 0x7f8810141380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35966/0 (socket says 172.21.15.5:35966) 2024-03-18T15:33:36.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.033+0000 7f880ffff640 1 -- 172.21.15.5:0/3813242369 learned_addr learned my addr 172.21.15.5:0/3813242369 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:33:36.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.033+0000 7f8815041640 1 --2- 172.21.15.5:0/3813242369 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f881014b330 0x7f8810151d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:36.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.033+0000 7f8814840640 1 -- 172.21.15.5:0/3813242369 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f881014b330 msgr2=0x7f8810151d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:36.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.033+0000 7f8814840640 1 --2- 172.21.15.5:0/3813242369 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f881014b330 0x7f8810151d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:36.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.033+0000 7f8814840640 1 -- 172.21.15.5:0/3813242369 >> v1:172.21.15.5:6789/0 conn(0x7f881006d9e0 legacy=0x7f881014ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:36.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.033+0000 7f8814840640 1 -- 172.21.15.5:0/3813242369 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88100b5e90 con 0x7f881006d600 2024-03-18T15:33:36.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.033+0000 7f8814840640 1 --2- 172.21.15.5:0/3813242369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f881006d600 0x7f8810141380 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f87fc0097c0 tx=0x7f87fc01c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d7c97a507c5435 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:36.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.033+0000 7f880f7fe640 1 -- 172.21.15.5:0/3813242369 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87fc0b0070 con 0x7f881006d600 2024-03-18T15:33:36.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.033+0000 7f880f7fe640 1 -- 172.21.15.5:0/3813242369 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87fc0ab600 con 0x7f881006d600 2024-03-18T15:33:36.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.033+0000 7f880f7fe640 1 -- 172.21.15.5:0/3813242369 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87fc0ab9d0 con 0x7f881006d600 2024-03-18T15:33:36.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.033+0000 7f8816aa4640 1 -- 172.21.15.5:0/3813242369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f881006d600 msgr2=0x7f8810141380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:36.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.033+0000 7f8816aa4640 1 --2- 172.21.15.5:0/3813242369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f881006d600 0x7f8810141380 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f87fc0097c0 tx=0x7f87fc01c380 comp rx=0 tx=0).stop 2024-03-18T15:33:36.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.033+0000 7f8816aa4640 1 -- 172.21.15.5:0/3813242369 shutdown_connections 2024-03-18T15:33:36.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.033+0000 7f8816aa4640 1 --2- 172.21.15.5:0/3813242369 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f881014b330 0x7f8810151d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:36.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.033+0000 7f8816aa4640 1 --2- 172.21.15.5:0/3813242369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f881006d600 0x7f8810141380 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:36.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.033+0000 7f8816aa4640 1 -- 172.21.15.5:0/3813242369 >> 172.21.15.5:0/3813242369 conn(0x7f88100bacb0 msgr2=0x7f88100bb0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:36.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.033+0000 7f8816aa4640 1 -- 172.21.15.5:0/3813242369 shutdown_connections 2024-03-18T15:33:36.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.033+0000 7f8816aa4640 1 -- 172.21.15.5:0/3813242369 wait complete. 2024-03-18T15:33:36.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.037+0000 7f8816aa4640 1 Processor -- start 2024-03-18T15:33:36.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.037+0000 7f8816aa4640 1 -- start start 2024-03-18T15:33:36.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.037+0000 7f8816aa4640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f881006d600 0x7f88101441a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:36.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.037+0000 7f8816aa4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f881006d9e0 0x7f88101e00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:36.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.037+0000 7f8816aa4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f881014b330 0x7f881013fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:36.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.037+0000 7f8816aa4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8810154e00 con 0x7f881014b330 2024-03-18T15:33:36.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.037+0000 7f8816aa4640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8810154c80 con 0x7f881006d9e0 2024-03-18T15:33:36.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.037+0000 7f8816aa4640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f8810154f80 con 0x7f881006d600 2024-03-18T15:33:36.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.037+0000 7f8815041640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f881014b330 0x7f881013fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:36.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.037+0000 7f8815041640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f881014b330 0x7f881013fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35976/0 (socket says 172.21.15.5:35976) 2024-03-18T15:33:36.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.037+0000 7f8815041640 1 -- 172.21.15.5:0/3750175676 learned_addr learned my addr 172.21.15.5:0/3750175676 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:36.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.037+0000 7f8814840640 1 --2- 172.21.15.5:0/3750175676 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f881006d600 0x7f88101441a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.037+0000 7f880ffff640 1 --2- 172.21.15.5:0/3750175676 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f881006d9e0 0x7f88101e00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.037+0000 7f8815041640 1 -- 172.21.15.5:0/3750175676 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f881006d600 msgr2=0x7f88101441a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.037+0000 7f8815041640 1 --2- 172.21.15.5:0/3750175676 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f881006d600 0x7f88101441a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.037+0000 7f8815041640 1 -- 172.21.15.5:0/3750175676 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f881006d9e0 msgr2=0x7f88101e00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.037+0000 7f8815041640 1 --2- 172.21.15.5:0/3750175676 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f881006d9e0 0x7f88101e00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.037+0000 7f8815041640 1 -- 172.21.15.5:0/3750175676 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88100adff0 con 0x7f881014b330 2024-03-18T15:33:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.037+0000 7f8815041640 1 --2- 172.21.15.5:0/3750175676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f881014b330 0x7f881013fdb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f880800efc0 tx=0x7f880800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.037+0000 7f880d7fa640 1 -- 172.21.15.5:0/3750175676 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8808018020 con 0x7f881014b330 2024-03-18T15:33:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.037+0000 7f880d7fa640 1 -- 172.21.15.5:0/3750175676 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8808014480 con 0x7f881014b330 2024-03-18T15:33:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.037+0000 7f8816aa4640 1 -- 172.21.15.5:0/3750175676 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88100ada00 con 0x7f881014b330 2024-03-18T15:33:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.037+0000 7f880d7fa640 1 -- 172.21.15.5:0/3750175676 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8808014870 con 0x7f881014b330 2024-03-18T15:33:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.037+0000 7f8816aa4640 1 -- 172.21.15.5:0/3750175676 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88100adc80 con 0x7f881014b330 2024-03-18T15:33:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.037+0000 7f8816aa4640 1 -- 172.21.15.5:0/3750175676 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f881004f550 con 0x7f881014b330 2024-03-18T15:33:36.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.037+0000 7f880d7fa640 1 -- 172.21.15.5:0/3750175676 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f880801b020 con 0x7f881014b330 2024-03-18T15:33:36.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.037+0000 7f880d7fa640 1 --2- 172.21.15.5:0/3750175676 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f87f4067660 0x7f87f4069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:36.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.037+0000 7f880d7fa640 1 -- 172.21.15.5:0/3750175676 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f8808086100 con 0x7f881014b330 2024-03-18T15:33:36.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.037+0000 7f8814840640 1 --2- 172.21.15.5:0/3750175676 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f87f4067660 0x7f87f4069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:36.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.041+0000 7f8814840640 1 --2- 172.21.15.5:0/3750175676 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f87f4067660 0x7f87f4069b20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f87fc0ad000 tx=0x7f87fc0a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:36.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.045+0000 7f880d7fa640 1 -- 172.21.15.5:0/3750175676 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f8808022030 con 0x7f881014b330 2024-03-18T15:33:36.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.205+0000 7f8816aa4640 1 -- 172.21.15.5:0/3750175676 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f88100aed40 con 0x7f881014b330 2024-03-18T15:33:36.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.205+0000 7f880d7fa640 1 -- 172.21.15.5:0/3750175676 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12728 (secure 0 0 0) 0x7f88100ada00 con 0x7f881014b330 2024-03-18T15:33:36.208 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:33:36.208 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":17,"fsid":"ef54fe03-5aa7-4eb2-9b1e-66c9cc345b94","created":"2024-03-18T15:33:18.086614+0000","modified":"2024-03-18T15:33:31.235982+0000","last_up_change":"2024-03-18T15:33:29.229911+0000","last_in_change":"2024-03-18T15:33:20.323840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:28.963186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df1c754b-af59-4d16-8919-fbb6386b3c48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6804","nonce":976264754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6808","nonce":976264754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6815","nonce":976264754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6811","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6813","nonce":976264754}]},"public_addr":"172.21.15.5:6804/976264754","cluster_addr":"172.21.15.5:6808/976264754","heartbeat_back_addr":"172.21.15.5:6815/976264754","heartbeat_front_addr":"172.21.15.5:6813/976264754","state":["exists","up"]},{"osd":1,"uuid":"b9a08b5e-8af4-4144-bdba-d69ed1ac627a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6801","nonce":3901761126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6803","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6805","nonce":3901761126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6812","nonce":3901761126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6807","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6809","nonce":3901761126}]},"public_addr":"172.21.15.5:6801/3901761126","cluster_addr":"172.21.15.5:6805/3901761126","heartbeat_back_addr":"172.21.15.5:6812/3901761126","heartbeat_front_addr":"172.21.15.5:6809/3901761126","state":["exists","up"]},{"osd":2,"uuid":"c3e8bb76-e382-439d-b380-62ef92f1255a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6817","nonce":502968448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6819","nonce":502968448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6823","nonce":502968448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6821","nonce":502968448}]},"public_addr":"172.21.15.5:6817/502968448","cluster_addr":"172.21.15.5:6819/502968448","heartbeat_back_addr":"172.21.15.5:6823/502968448","heartbeat_front_addr":"172.21.15.5:6821/502968448","state":["exists","up"]},{"osd":3,"uuid":"e8c94625-f2cd-4eba-b53d-2e7356e1c02a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6825","nonce":2425932735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6827","nonce":2425932735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6831","nonce":2425932735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6829","nonce":2425932735}]},"public_addr":"172.21.15.5:6825/2425932735","cluster_addr":"172.21.15.5:6827/2425932735","heartbeat_back_addr":"172.21.15.5:6831/2425932735","heartbeat_front_addr":"172.21.15.5:6829/2425932735","state":["exists","up"]},{"osd":4,"uuid":"0099dd52-6265-4d32-82e4-3ef16e73add0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6817","nonce":976514285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6819","nonce":976514285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6823","nonce":976514285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6821","nonce":976514285}]},"public_addr":"172.21.15.80:6817/976514285","cluster_addr":"172.21.15.80:6819/976514285","heartbeat_back_addr":"172.21.15.80:6823/976514285","heartbeat_front_addr":"172.21.15.80:6821/976514285","state":["exists","up"]},{"osd":5,"uuid":"ed10c84f-8c39-41cc-9b64-7c482a94a48e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6825","nonce":627153447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6827","nonce":627153447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6831","nonce":627153447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6829","nonce":627153447}]},"public_addr":"172.21.15.80:6825/627153447","cluster_addr":"172.21.15.80:6827/627153447","heartbeat_back_addr":"172.21.15.80:6831/627153447","heartbeat_front_addr":"172.21.15.80:6829/627153447","state":["exists","up"]},{"osd":6,"uuid":"c7a3538a-7048-4fba-8781-b440f03c25db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6803","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6805","nonce":488342527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6807","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6809","nonce":488342527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6815","nonce":488342527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6811","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6813","nonce":488342527}]},"public_addr":"172.21.15.80:6805/488342527","cluster_addr":"172.21.15.80:6809/488342527","heartbeat_back_addr":"172.21.15.80:6815/488342527","heartbeat_front_addr":"172.21.15.80:6813/488342527","state":["exists","up"]},{"osd":7,"uuid":"f9418104-3a97-4110-bc3a-1f8151d0f757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6801","nonce":982831666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6804","nonce":982831666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6812","nonce":982831666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6808","nonce":982831666}]},"public_addr":"172.21.15.80:6801/982831666","cluster_addr":"172.21.15.80:6804/982831666","heartbeat_back_addr":"172.21.15.80:6812/982831666","heartbeat_front_addr":"172.21.15.80:6808/982831666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.970828+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.858116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.987556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.864603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.756293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.591502+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:33:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.209+0000 7f8816aa4640 1 -- 172.21.15.5:0/3750175676 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f87f4067660 msgr2=0x7f87f4069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.209+0000 7f8816aa4640 1 --2- 172.21.15.5:0/3750175676 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f87f4067660 0x7f87f4069b20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f87fc0ad000 tx=0x7f87fc0a6000 comp rx=0 tx=0).stop 2024-03-18T15:33:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.209+0000 7f8816aa4640 1 -- 172.21.15.5:0/3750175676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f881014b330 msgr2=0x7f881013fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.209+0000 7f8816aa4640 1 --2- 172.21.15.5:0/3750175676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f881014b330 0x7f881013fdb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f880800efc0 tx=0x7f880800c490 comp rx=0 tx=0).stop 2024-03-18T15:33:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.209+0000 7f8816aa4640 1 -- 172.21.15.5:0/3750175676 shutdown_connections 2024-03-18T15:33:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.209+0000 7f8816aa4640 1 --2- 172.21.15.5:0/3750175676 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f87f4067660 0x7f87f4069b20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.209+0000 7f8816aa4640 1 --2- 172.21.15.5:0/3750175676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f881014b330 0x7f881013fdb0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.209+0000 7f8816aa4640 1 --2- 172.21.15.5:0/3750175676 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f881006d9e0 0x7f88101e00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.209+0000 7f8816aa4640 1 --2- 172.21.15.5:0/3750175676 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f881006d600 0x7f88101441a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.209+0000 7f8816aa4640 1 -- 172.21.15.5:0/3750175676 >> 172.21.15.5:0/3750175676 conn(0x7f88100bacb0 msgr2=0x7f88100b7f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.209+0000 7f8816aa4640 1 -- 172.21.15.5:0/3750175676 shutdown_connections 2024-03-18T15:33:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.209+0000 7f8816aa4640 1 -- 172.21.15.5:0/3750175676 wait complete. 2024-03-18T15:33:36.229 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T15:33:28.963186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T15:33:36.229 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T15:33:36.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.357+0000 7f75b6b71640 1 Processor -- start 2024-03-18T15:33:36.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.357+0000 7f75b6b71640 1 -- start start 2024-03-18T15:33:36.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.357+0000 7f75b6b71640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f75b00af790 0x7f75b0072b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:36.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.357+0000 7f75b6b71640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75b00afb70 con 0x7f75b006d070 2024-03-18T15:33:36.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.357+0000 7f75b6b71640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75b00b4a20 con 0x7f75b00b26a0 2024-03-18T15:33:36.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.357+0000 7f75b6b71640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f75b00b4d00 con 0x7f75b00af790 2024-03-18T15:33:36.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.357+0000 7f75affff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f75b006d070 0x7f75b0147780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:39356/0 (socket says 172.21.15.5:39356) 2024-03-18T15:33:36.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.357+0000 7f75affff640 1 -- 172.21.15.5:0/527723663 learned_addr learned my addr 172.21.15.5:0/527723663 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:33:36.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.357+0000 7f75b490d640 1 --2- 172.21.15.5:0/527723663 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f75b00af790 0x7f75b0072b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:36.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.357+0000 7f75af7fe640 1 -- 172.21.15.5:0/527723663 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2383215735 0 0) 0x7f75b00afb70 con 0x7f75b006d070 2024-03-18T15:33:36.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.357+0000 7f75af7fe640 1 -- 172.21.15.5:0/527723663 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f75a0005740 con 0x7f75b006d070 2024-03-18T15:33:36.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.357+0000 7f75af7fe640 1 -- 172.21.15.5:0/527723663 <== mon.1 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 185843234 0 0) 0x7f75b00b4a20 con 0x7f75b00b26a0 2024-03-18T15:33:36.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.357+0000 7f75b490d640 1 -- 172.21.15.5:0/527723663 >> v1:172.21.15.80:6789/0 conn(0x7f75b00b26a0 legacy=0x7f75b00b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:36.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.357+0000 7f75b490d640 1 -- 172.21.15.5:0/527723663 >> v1:172.21.15.5:6789/0 conn(0x7f75b006d070 legacy=0x7f75b0147780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:36.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.357+0000 7f75b490d640 1 -- 172.21.15.5:0/527723663 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b006d4b0 con 0x7f75b00af790 2024-03-18T15:33:36.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.361+0000 7f75b490d640 1 --2- 172.21.15.5:0/527723663 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f75b00af790 0x7f75b0072b40 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f75a40097f0 tx=0x7f75a401c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a07dff86de439de server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:36.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.361+0000 7f75af7fe640 1 -- 172.21.15.5:0/527723663 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a40b0070 con 0x7f75b00af790 2024-03-18T15:33:36.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.361+0000 7f75af7fe640 1 -- 172.21.15.5:0/527723663 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75a40ab600 con 0x7f75b00af790 2024-03-18T15:33:36.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.361+0000 7f75af7fe640 1 -- 172.21.15.5:0/527723663 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a40ab9b0 con 0x7f75b00af790 2024-03-18T15:33:36.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.361+0000 7f75b6b71640 1 -- 172.21.15.5:0/527723663 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f75b00af790 msgr2=0x7f75b0072b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:36.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.361+0000 7f75b6b71640 1 --2- 172.21.15.5:0/527723663 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f75b00af790 0x7f75b0072b40 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f75a40097f0 tx=0x7f75a401c450 comp rx=0 tx=0).stop 2024-03-18T15:33:36.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.361+0000 7f75b6b71640 1 -- 172.21.15.5:0/527723663 shutdown_connections 2024-03-18T15:33:36.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.361+0000 7f75b6b71640 1 --2- 172.21.15.5:0/527723663 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f75b00af790 0x7f75b0072b40 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:36.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.361+0000 7f75b6b71640 1 -- 172.21.15.5:0/527723663 >> 172.21.15.5:0/527723663 conn(0x7f75b00b5670 msgr2=0x7f75b00b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:36.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.361+0000 7f75b6b71640 1 -- 172.21.15.5:0/527723663 shutdown_connections 2024-03-18T15:33:36.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.361+0000 7f75b6b71640 1 -- 172.21.15.5:0/527723663 wait complete. 2024-03-18T15:33:36.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.361+0000 7f75b6b71640 1 Processor -- start 2024-03-18T15:33:36.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.361+0000 7f75b6b71640 1 -- start start 2024-03-18T15:33:36.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.361+0000 7f75b6b71640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f75b006d070 0x7f75b0144820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:36.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.361+0000 7f75b6b71640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75b00af790 0x7f75b01df580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:36.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.361+0000 7f75b6b71640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f75b00b26a0 0x7f75b01428c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:36.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.361+0000 7f75b6b71640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f75b00b4110 con 0x7f75b00af790 2024-03-18T15:33:36.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.361+0000 7f75b6b71640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f75b00b3f90 con 0x7f75b00b26a0 2024-03-18T15:33:36.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.361+0000 7f75b6b71640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f75b00b4290 con 0x7f75b006d070 2024-03-18T15:33:36.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.361+0000 7f75affff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75b00af790 0x7f75b01df580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:36.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.361+0000 7f75affff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75b00af790 0x7f75b01df580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35986/0 (socket says 172.21.15.5:35986) 2024-03-18T15:33:36.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.361+0000 7f75affff640 1 -- 172.21.15.5:0/1836109076 learned_addr learned my addr 172.21.15.5:0/1836109076 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:36.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.361+0000 7f75b510e640 1 --2- 172.21.15.5:0/1836109076 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f75b00b26a0 0x7f75b01428c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:36.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.365+0000 7f75b490d640 1 --2- 172.21.15.5:0/1836109076 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f75b006d070 0x7f75b0144820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:36.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.365+0000 7f75affff640 1 -- 172.21.15.5:0/1836109076 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f75b006d070 msgr2=0x7f75b0144820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:36.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.365+0000 7f75affff640 1 --2- 172.21.15.5:0/1836109076 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f75b006d070 0x7f75b0144820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:36.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.365+0000 7f75affff640 1 -- 172.21.15.5:0/1836109076 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f75b00b26a0 msgr2=0x7f75b01428c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:36.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.365+0000 7f75affff640 1 --2- 172.21.15.5:0/1836109076 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f75b00b26a0 0x7f75b01428c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:36.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.365+0000 7f75affff640 1 -- 172.21.15.5:0/1836109076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b014ca90 con 0x7f75b00af790 2024-03-18T15:33:36.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.365+0000 7f75affff640 1 --2- 172.21.15.5:0/1836109076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75b00af790 0x7f75b01df580 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f759c00c3c0 tx=0x7f759c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:36.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.365+0000 7f75ad7fa640 1 -- 172.21.15.5:0/1836109076 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759c017070 con 0x7f75b00af790 2024-03-18T15:33:36.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.365+0000 7f75ad7fa640 1 -- 172.21.15.5:0/1836109076 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f759c005cf0 con 0x7f75b00af790 2024-03-18T15:33:36.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.365+0000 7f75b6b71640 1 -- 172.21.15.5:0/1836109076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75b01dfdd0 con 0x7f75b00af790 2024-03-18T15:33:36.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.365+0000 7f75ad7fa640 1 -- 172.21.15.5:0/1836109076 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759c0072f0 con 0x7f75b00af790 2024-03-18T15:33:36.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.365+0000 7f75b6b71640 1 -- 172.21.15.5:0/1836109076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75b0144d60 con 0x7f75b00af790 2024-03-18T15:33:36.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.365+0000 7f75b6b71640 1 -- 172.21.15.5:0/1836109076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75b01425b0 con 0x7f75b00af790 2024-03-18T15:33:36.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.365+0000 7f75ad7fa640 1 -- 172.21.15.5:0/1836109076 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f759c01a070 con 0x7f75b00af790 2024-03-18T15:33:36.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.365+0000 7f75ad7fa640 1 --2- 172.21.15.5:0/1836109076 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f7594067660 0x7f7594069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:36.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.365+0000 7f75ad7fa640 1 -- 172.21.15.5:0/1836109076 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f759c0848b0 con 0x7f75b00af790 2024-03-18T15:33:36.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.365+0000 7f75b490d640 1 --2- 172.21.15.5:0/1836109076 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f7594067660 0x7f7594069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:36.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.365+0000 7f75b490d640 1 --2- 172.21.15.5:0/1836109076 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f7594067660 0x7f7594069b20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f75a40ad000 tx=0x7f75a40a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:36.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.369+0000 7f75ad7fa640 1 -- 172.21.15.5:0/1836109076 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f759c055240 con 0x7f75b00af790 2024-03-18T15:33:36.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.525+0000 7f75b6b71640 1 -- 172.21.15.5:0/1836109076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f75b004f550 con 0x7f75b00af790 2024-03-18T15:33:36.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.525+0000 7f75ad7fa640 1 -- 172.21.15.5:0/1836109076 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) ==== 93+0+10 (secure 0 0 0) 0x7f759c0592e0 con 0x7f75b00af790 2024-03-18T15:33:36.526 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-03-18T15:33:36.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.525+0000 7f75b6b71640 1 -- 172.21.15.5:0/1836109076 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f7594067660 msgr2=0x7f7594069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:36.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.525+0000 7f75b6b71640 1 --2- 172.21.15.5:0/1836109076 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f7594067660 0x7f7594069b20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f75a40ad000 tx=0x7f75a40a6000 comp rx=0 tx=0).stop 2024-03-18T15:33:36.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.525+0000 7f75b6b71640 1 -- 172.21.15.5:0/1836109076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75b00af790 msgr2=0x7f75b01df580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:36.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.525+0000 7f75b6b71640 1 --2- 172.21.15.5:0/1836109076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75b00af790 0x7f75b01df580 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f759c00c3c0 tx=0x7f759c00c880 comp rx=0 tx=0).stop 2024-03-18T15:33:36.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.525+0000 7f75b6b71640 1 -- 172.21.15.5:0/1836109076 shutdown_connections 2024-03-18T15:33:36.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.525+0000 7f75b6b71640 1 --2- 172.21.15.5:0/1836109076 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f7594067660 0x7f7594069b20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:36.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.525+0000 7f75b6b71640 1 --2- 172.21.15.5:0/1836109076 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f75b00b26a0 0x7f75b01428c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:36.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.525+0000 7f75b6b71640 1 --2- 172.21.15.5:0/1836109076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75b00af790 0x7f75b01df580 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:36.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.525+0000 7f75b6b71640 1 --2- 172.21.15.5:0/1836109076 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f75b006d070 0x7f75b0144820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:36.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.525+0000 7f75b6b71640 1 -- 172.21.15.5:0/1836109076 >> 172.21.15.5:0/1836109076 conn(0x7f75b00b5670 msgr2=0x7f75b006c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:36.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.525+0000 7f75b6b71640 1 -- 172.21.15.5:0/1836109076 shutdown_connections 2024-03-18T15:33:36.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.525+0000 7f75b6b71640 1 -- 172.21.15.5:0/1836109076 wait complete. 2024-03-18T15:33:36.544 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:33:36.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.653+0000 7f97d2c2a640 1 Processor -- start 2024-03-18T15:33:36.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.653+0000 7f97d2c2a640 1 -- start start 2024-03-18T15:33:36.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.657+0000 7f97d2c2a640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f97cc13f260 0x7f97cc13f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:36.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.657+0000 7f97d2c2a640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f97cc14dc70 0x7f97cc150060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:36.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.657+0000 7f97d2c2a640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97cc0b2ab0 con 0x7f97cc06d2d0 2024-03-18T15:33:36.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.657+0000 7f97d2c2a640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f97cc0728c0 con 0x7f97cc14dc70 2024-03-18T15:33:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.657+0000 7f97d2c2a640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f97cc0b0ea0 con 0x7f97cc13f260 2024-03-18T15:33:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.657+0000 7f97d11c7640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f97cc14dc70 0x7f97cc150060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.657+0000 7f97d11c7640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f97cc14dc70 0x7f97cc150060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35996/0 (socket says 172.21.15.5:35996) 2024-03-18T15:33:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.657+0000 7f97d11c7640 1 -- 172.21.15.5:0/4211539575 learned_addr learned my addr 172.21.15.5:0/4211539575 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.657+0000 7f97cbfff640 1 --2- 172.21.15.5:0/4211539575 >> v2:172.21.15.80:3300/0 conn(0x7f97cc13f260 0x7f97cc13f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.657+0000 7f97d11c7640 1 -- 172.21.15.5:0/4211539575 >> v2:172.21.15.80:3300/0 conn(0x7f97cc13f260 msgr2=0x7f97cc13f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.657+0000 7f97d11c7640 1 --2- 172.21.15.5:0/4211539575 >> v2:172.21.15.80:3300/0 conn(0x7f97cc13f260 0x7f97cc13f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.657+0000 7f97d11c7640 1 -- 172.21.15.5:0/4211539575 >> v1:172.21.15.80:6789/0 conn(0x7f97cc06d2d0 legacy=0x7f97cc0b23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.657+0000 7f97d11c7640 1 -- 172.21.15.5:0/4211539575 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97cc06d6b0 con 0x7f97cc14dc70 2024-03-18T15:33:36.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.657+0000 7f97d11c7640 1 --2- 172.21.15.5:0/4211539575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97cc14dc70 0x7f97cc150060 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f97cc09eaa0 tx=0x7f97c401c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2103d82482e5310 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:36.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.657+0000 7f97cb7fe640 1 -- 172.21.15.5:0/4211539575 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c40a56d0 con 0x7f97cc14dc70 2024-03-18T15:33:36.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.657+0000 7f97cb7fe640 1 -- 172.21.15.5:0/4211539575 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97c40a5870 con 0x7f97cc14dc70 2024-03-18T15:33:36.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.657+0000 7f97cb7fe640 1 -- 172.21.15.5:0/4211539575 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c40a5c40 con 0x7f97cc14dc70 2024-03-18T15:33:36.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.661+0000 7f97d2c2a640 1 -- 172.21.15.5:0/4211539575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97cc14dc70 msgr2=0x7f97cc150060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:36.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.661+0000 7f97d2c2a640 1 --2- 172.21.15.5:0/4211539575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97cc14dc70 0x7f97cc150060 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f97cc09eaa0 tx=0x7f97c401c920 comp rx=0 tx=0).stop 2024-03-18T15:33:36.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.661+0000 7f97d2c2a640 1 -- 172.21.15.5:0/4211539575 shutdown_connections 2024-03-18T15:33:36.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.661+0000 7f97d2c2a640 1 --2- 172.21.15.5:0/4211539575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97cc14dc70 0x7f97cc150060 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:36.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.661+0000 7f97d2c2a640 1 --2- 172.21.15.5:0/4211539575 >> v2:172.21.15.80:3300/0 conn(0x7f97cc13f260 0x7f97cc13f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:36.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.661+0000 7f97d2c2a640 1 -- 172.21.15.5:0/4211539575 >> 172.21.15.5:0/4211539575 conn(0x7f97cc0b5670 msgr2=0x7f97cc0b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:36.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.661+0000 7f97d2c2a640 1 -- 172.21.15.5:0/4211539575 shutdown_connections 2024-03-18T15:33:36.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.661+0000 7f97d2c2a640 1 -- 172.21.15.5:0/4211539575 wait complete. 2024-03-18T15:33:36.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.661+0000 7f97d2c2a640 1 Processor -- start 2024-03-18T15:33:36.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.661+0000 7f97d2c2a640 1 -- start start 2024-03-18T15:33:36.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.661+0000 7f97d2c2a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97cc06d2d0 0x7f97cc1449e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:36.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.661+0000 7f97d2c2a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97cc13f260 0x7f97cc1dffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:36.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.661+0000 7f97d2c2a640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f97cc14dc70 0x7f97cc1e0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:36.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.661+0000 7f97d2c2a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f97cc0b0960 con 0x7f97cc13f260 2024-03-18T15:33:36.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.661+0000 7f97d2c2a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f97cc0b07e0 con 0x7f97cc06d2d0 2024-03-18T15:33:36.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.661+0000 7f97d2c2a640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f97cc072d50 con 0x7f97cc14dc70 2024-03-18T15:33:36.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.661+0000 7f97cbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97cc13f260 0x7f97cc1dffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:36.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.661+0000 7f97cbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97cc13f260 0x7f97cc1dffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36010/0 (socket says 172.21.15.5:36010) 2024-03-18T15:33:36.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.661+0000 7f97cbfff640 1 -- 172.21.15.5:0/3135682032 learned_addr learned my addr 172.21.15.5:0/3135682032 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:36.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.661+0000 7f97d09c6640 1 --2- 172.21.15.5:0/3135682032 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97cc06d2d0 0x7f97cc1449e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:36.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.661+0000 7f97d11c7640 1 --2- 172.21.15.5:0/3135682032 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f97cc14dc70 0x7f97cc1e0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:36.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.661+0000 7f97cbfff640 1 -- 172.21.15.5:0/3135682032 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f97cc14dc70 msgr2=0x7f97cc1e0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:36.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.661+0000 7f97cbfff640 1 --2- 172.21.15.5:0/3135682032 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f97cc14dc70 0x7f97cc1e0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:36.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.661+0000 7f97cbfff640 1 -- 172.21.15.5:0/3135682032 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97cc06d2d0 msgr2=0x7f97cc1449e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:36.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.661+0000 7f97cbfff640 1 --2- 172.21.15.5:0/3135682032 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97cc06d2d0 0x7f97cc1449e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:36.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.661+0000 7f97cbfff640 1 -- 172.21.15.5:0/3135682032 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97cc14b800 con 0x7f97cc13f260 2024-03-18T15:33:36.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.661+0000 7f97cbfff640 1 --2- 172.21.15.5:0/3135682032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97cc13f260 0x7f97cc1dffe0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f97bc00d830 tx=0x7f97bc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:36.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.661+0000 7f97c97fa640 1 -- 172.21.15.5:0/3135682032 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97bc00b6c0 con 0x7f97cc13f260 2024-03-18T15:33:36.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.661+0000 7f97c97fa640 1 -- 172.21.15.5:0/3135682032 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97bc019070 con 0x7f97cc13f260 2024-03-18T15:33:36.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.661+0000 7f97d2c2a640 1 -- 172.21.15.5:0/3135682032 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97cc146090 con 0x7f97cc13f260 2024-03-18T15:33:36.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.661+0000 7f97c97fa640 1 -- 172.21.15.5:0/3135682032 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97bc00ba60 con 0x7f97cc13f260 2024-03-18T15:33:36.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.661+0000 7f97d2c2a640 1 -- 172.21.15.5:0/3135682032 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97cc0af950 con 0x7f97cc13f260 2024-03-18T15:33:36.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.661+0000 7f97d2c2a640 1 -- 172.21.15.5:0/3135682032 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97cc04f550 con 0x7f97cc13f260 2024-03-18T15:33:36.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.665+0000 7f97c97fa640 1 -- 172.21.15.5:0/3135682032 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f97bc01e460 con 0x7f97cc13f260 2024-03-18T15:33:36.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.665+0000 7f97c97fa640 1 --2- 172.21.15.5:0/3135682032 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f97b8067660 0x7f97b8069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:36.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.665+0000 7f97d09c6640 1 --2- 172.21.15.5:0/3135682032 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f97b8067660 0x7f97b8069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:36.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.665+0000 7f97c97fa640 1 -- 172.21.15.5:0/3135682032 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f97bc085b70 con 0x7f97cc13f260 2024-03-18T15:33:36.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.665+0000 7f97d09c6640 1 --2- 172.21.15.5:0/3135682032 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f97b8067660 0x7f97b8069b20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f97c0004e30 tx=0x7f97c0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:36.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.669+0000 7f97c97fa640 1 -- 172.21.15.5:0/3135682032 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f97bc056bd0 con 0x7f97cc13f260 2024-03-18T15:33:36.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.861+0000 7f97d2c2a640 1 -- 172.21.15.5:0/3135682032 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97cc13dbc0 con 0x7f97cc13f260 2024-03-18T15:33:36.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.865+0000 7f97c97fa640 1 -- 172.21.15.5:0/3135682032 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1386 (secure 0 0 0) 0x7f97bc0564d0 con 0x7f97cc13f260 2024-03-18T15:33:36.866 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:33:36.866 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/629589896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":629589896},{"type":"v1","addr":"172.21.15.5:6835","nonce":629589896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-03-18T15:33:36.866 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 2 2024-03-18T15:33:36.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.865+0000 7f97d2c2a640 1 -- 172.21.15.5:0/3135682032 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f97b8067660 msgr2=0x7f97b8069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:36.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.865+0000 7f97d2c2a640 1 --2- 172.21.15.5:0/3135682032 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f97b8067660 0x7f97b8069b20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f97c0004e30 tx=0x7f97c0009040 comp rx=0 tx=0).stop 2024-03-18T15:33:36.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.865+0000 7f97d2c2a640 1 -- 172.21.15.5:0/3135682032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97cc13f260 msgr2=0x7f97cc1dffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:36.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.865+0000 7f97d2c2a640 1 --2- 172.21.15.5:0/3135682032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97cc13f260 0x7f97cc1dffe0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f97bc00d830 tx=0x7f97bc00dcf0 comp rx=0 tx=0).stop 2024-03-18T15:33:36.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.865+0000 7f97d2c2a640 1 -- 172.21.15.5:0/3135682032 shutdown_connections 2024-03-18T15:33:36.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.865+0000 7f97d2c2a640 1 --2- 172.21.15.5:0/3135682032 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f97b8067660 0x7f97b8069b20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:36.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.865+0000 7f97d2c2a640 1 --2- 172.21.15.5:0/3135682032 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f97cc14dc70 0x7f97cc1e0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:36.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.865+0000 7f97d2c2a640 1 --2- 172.21.15.5:0/3135682032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97cc13f260 0x7f97cc1dffe0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:36.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.865+0000 7f97d2c2a640 1 --2- 172.21.15.5:0/3135682032 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97cc06d2d0 0x7f97cc1449e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:36.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.865+0000 7f97d2c2a640 1 -- 172.21.15.5:0/3135682032 >> 172.21.15.5:0/3135682032 conn(0x7f97cc0b5670 msgr2=0x7f97cc13e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:36.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.865+0000 7f97d2c2a640 1 -- 172.21.15.5:0/3135682032 shutdown_connections 2024-03-18T15:33:36.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:36.865+0000 7f97d2c2a640 1 -- 172.21.15.5:0/3135682032 wait complete. 2024-03-18T15:33:38.884 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:33:39.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.017+0000 7fe3c5151640 1 Processor -- start 2024-03-18T15:33:39.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.017+0000 7fe3c5151640 1 -- start start 2024-03-18T15:33:39.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.021+0000 7fe3c5151640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe3c006d2d0 0x7fe3c00b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:39.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.021+0000 7fe3c5151640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe3c013f260 0x7fe3c013f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:39.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.021+0000 7fe3c5151640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3c00b0ea0 con 0x7fe3c01453a0 2024-03-18T15:33:39.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.021+0000 7fe3c5151640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fe3c00728c0 con 0x7fe3c006d2d0 2024-03-18T15:33:39.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.021+0000 7fe3c5151640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fe3c006d6b0 con 0x7fe3c013f260 2024-03-18T15:33:39.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.021+0000 7fe3bed9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe3c006d2d0 0x7fe3c00b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:39.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.021+0000 7fe3bed9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe3c006d2d0 0x7fe3c00b23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36014/0 (socket says 172.21.15.5:36014) 2024-03-18T15:33:39.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.021+0000 7fe3bed9d640 1 -- 172.21.15.5:0/3551786235 learned_addr learned my addr 172.21.15.5:0/3551786235 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:39.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.021+0000 7fe3bed9d640 1 -- 172.21.15.5:0/3551786235 >> v2:172.21.15.80:3300/0 conn(0x7fe3c013f260 msgr2=0x7fe3c013f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:39.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.021+0000 7fe3bed9d640 1 --2- 172.21.15.5:0/3551786235 >> v2:172.21.15.80:3300/0 conn(0x7fe3c013f260 0x7fe3c013f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:39.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.021+0000 7fe3bed9d640 1 -- 172.21.15.5:0/3551786235 >> v1:172.21.15.5:6789/0 conn(0x7fe3c01453a0 legacy=0x7fe3c01457f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:39.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.021+0000 7fe3bed9d640 1 -- 172.21.15.5:0/3551786235 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3c00b0bc0 con 0x7fe3c006d2d0 2024-03-18T15:33:39.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.021+0000 7fe3bed9d640 1 --2- 172.21.15.5:0/3551786235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3c006d2d0 0x7fe3c00b23a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe3c009eaa0 tx=0x7fe3b001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3793a69c1ef0b966 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:39.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.021+0000 7fe3bdd9b640 1 -- 172.21.15.5:0/3551786235 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b00b0070 con 0x7fe3c006d2d0 2024-03-18T15:33:39.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.021+0000 7fe3bdd9b640 1 -- 172.21.15.5:0/3551786235 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3b00ab600 con 0x7fe3c006d2d0 2024-03-18T15:33:39.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.021+0000 7fe3bdd9b640 1 -- 172.21.15.5:0/3551786235 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b00ab9d0 con 0x7fe3c006d2d0 2024-03-18T15:33:39.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.025+0000 7fe3c5151640 1 -- 172.21.15.5:0/3551786235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3c006d2d0 msgr2=0x7fe3c00b23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:39.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.025+0000 7fe3c5151640 1 --2- 172.21.15.5:0/3551786235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3c006d2d0 0x7fe3c00b23a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe3c009eaa0 tx=0x7fe3b001c490 comp rx=0 tx=0).stop 2024-03-18T15:33:39.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.025+0000 7fe3c5151640 1 -- 172.21.15.5:0/3551786235 shutdown_connections 2024-03-18T15:33:39.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.025+0000 7fe3c5151640 1 --2- 172.21.15.5:0/3551786235 >> v2:172.21.15.80:3300/0 conn(0x7fe3c013f260 0x7fe3c013f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:39.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.025+0000 7fe3c5151640 1 --2- 172.21.15.5:0/3551786235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3c006d2d0 0x7fe3c00b23a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:39.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.025+0000 7fe3c5151640 1 -- 172.21.15.5:0/3551786235 >> 172.21.15.5:0/3551786235 conn(0x7fe3c00b5670 msgr2=0x7fe3c00b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:39.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.025+0000 7fe3c5151640 1 -- 172.21.15.5:0/3551786235 shutdown_connections 2024-03-18T15:33:39.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.025+0000 7fe3c5151640 1 -- 172.21.15.5:0/3551786235 wait complete. 2024-03-18T15:33:39.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.025+0000 7fe3c5151640 1 Processor -- start 2024-03-18T15:33:39.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.025+0000 7fe3c5151640 1 -- start start 2024-03-18T15:33:39.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.025+0000 7fe3c5151640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3c006d2d0 0x7fe3c0142140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:39.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.025+0000 7fe3c5151640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3c013f260 0x7fe3c0137bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:39.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.025+0000 7fe3c5151640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe3c01453a0 0x7fe3c0138110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:39.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.025+0000 7fe3c5151640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c0072ed0 con 0x7fe3c006d2d0 2024-03-18T15:33:39.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.025+0000 7fe3c5151640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c0072d50 con 0x7fe3c013f260 2024-03-18T15:33:39.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.025+0000 7fe3c5151640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fe3c00b5140 con 0x7fe3c01453a0 2024-03-18T15:33:39.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.025+0000 7fe3bed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3c006d2d0 0x7fe3c0142140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:39.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.025+0000 7fe3bed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3c006d2d0 0x7fe3c0142140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36026/0 (socket says 172.21.15.5:36026) 2024-03-18T15:33:39.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.025+0000 7fe3bed9d640 1 -- 172.21.15.5:0/2573207330 learned_addr learned my addr 172.21.15.5:0/2573207330 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:39.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.025+0000 7fe3bed9d640 1 -- 172.21.15.5:0/2573207330 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe3c01453a0 msgr2=0x7fe3c0138110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:39.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.025+0000 7fe3bed9d640 1 --2- 172.21.15.5:0/2573207330 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe3c01453a0 0x7fe3c0138110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:39.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.025+0000 7fe3bed9d640 1 -- 172.21.15.5:0/2573207330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3c013f260 msgr2=0x7fe3c0137bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:39.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.025+0000 7fe3be59c640 1 --2- 172.21.15.5:0/2573207330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3c013f260 0x7fe3c0137bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:39.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.025+0000 7fe3bed9d640 1 --2- 172.21.15.5:0/2573207330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3c013f260 0x7fe3c0137bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:39.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.025+0000 7fe3bed9d640 1 -- 172.21.15.5:0/2573207330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3c00713c0 con 0x7fe3c006d2d0 2024-03-18T15:33:39.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.025+0000 7fe3bed9d640 1 --2- 172.21.15.5:0/2573207330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3c006d2d0 0x7fe3c0142140 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe3b00ae040 tx=0x7fe3b001acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:39.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.025+0000 7fe3a77fe640 1 -- 172.21.15.5:0/2573207330 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b00ba020 con 0x7fe3c006d2d0 2024-03-18T15:33:39.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.025+0000 7fe3a77fe640 1 -- 172.21.15.5:0/2573207330 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3b00b6620 con 0x7fe3c006d2d0 2024-03-18T15:33:39.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.025+0000 7fe3a77fe640 1 -- 172.21.15.5:0/2573207330 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b00b0040 con 0x7fe3c006d2d0 2024-03-18T15:33:39.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.025+0000 7fe3c5151640 1 -- 172.21.15.5:0/2573207330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3c01394e0 con 0x7fe3c006d2d0 2024-03-18T15:33:39.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.025+0000 7fe3c5151640 1 -- 172.21.15.5:0/2573207330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3c0071660 con 0x7fe3c006d2d0 2024-03-18T15:33:39.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.029+0000 7fe3c5151640 1 -- 172.21.15.5:0/2573207330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3c004f550 con 0x7fe3c006d2d0 2024-03-18T15:33:39.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.029+0000 7fe3a77fe640 1 -- 172.21.15.5:0/2573207330 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fe3b00bd020 con 0x7fe3c006d2d0 2024-03-18T15:33:39.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.029+0000 7fe3a77fe640 1 --2- 172.21.15.5:0/2573207330 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe3ac067660 0x7fe3ac069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:39.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.029+0000 7fe3a77fe640 1 -- 172.21.15.5:0/2573207330 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fe3b011f920 con 0x7fe3c006d2d0 2024-03-18T15:33:39.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.029+0000 7fe3be59c640 1 --2- 172.21.15.5:0/2573207330 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe3ac067660 0x7fe3ac069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:39.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.029+0000 7fe3be59c640 1 --2- 172.21.15.5:0/2573207330 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe3ac067660 0x7fe3ac069b20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0150af0 tx=0x7fe3b4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:39.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.033+0000 7fe3a77fe640 1 -- 172.21.15.5:0/2573207330 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fe3b00c4030 con 0x7fe3c006d2d0 2024-03-18T15:33:39.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.229+0000 7fe3c5151640 1 -- 172.21.15.5:0/2573207330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe3c0142d40 con 0x7fe3c006d2d0 2024-03-18T15:33:39.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.229+0000 7fe3a77fe640 1 -- 172.21.15.5:0/2573207330 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2210 (secure 0 0 0) 0x7fe3b0016110 con 0x7fe3c006d2d0 2024-03-18T15:33:39.231 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:33:39.231 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/629589896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":629589896},{"type":"v1","addr":"172.21.15.5:6835","nonce":629589896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4272,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/1236603350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":1236603350},{"type":"v1","addr":"172.21.15.80:6833","nonce":1236603350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-03-18T15:33:39.231 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 3 2024-03-18T15:33:39.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.229+0000 7fe3c5151640 1 -- 172.21.15.5:0/2573207330 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe3ac067660 msgr2=0x7fe3ac069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:39.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.229+0000 7fe3c5151640 1 --2- 172.21.15.5:0/2573207330 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe3ac067660 0x7fe3ac069b20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0150af0 tx=0x7fe3b4008040 comp rx=0 tx=0).stop 2024-03-18T15:33:39.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.229+0000 7fe3c5151640 1 -- 172.21.15.5:0/2573207330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3c006d2d0 msgr2=0x7fe3c0142140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:39.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.229+0000 7fe3c5151640 1 --2- 172.21.15.5:0/2573207330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3c006d2d0 0x7fe3c0142140 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe3b00ae040 tx=0x7fe3b001acd0 comp rx=0 tx=0).stop 2024-03-18T15:33:39.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.229+0000 7fe3c5151640 1 -- 172.21.15.5:0/2573207330 shutdown_connections 2024-03-18T15:33:39.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.229+0000 7fe3c5151640 1 --2- 172.21.15.5:0/2573207330 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe3ac067660 0x7fe3ac069b20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:39.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.229+0000 7fe3c5151640 1 --2- 172.21.15.5:0/2573207330 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe3c01453a0 0x7fe3c0138110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:39.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.229+0000 7fe3c5151640 1 --2- 172.21.15.5:0/2573207330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3c013f260 0x7fe3c0137bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:39.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.229+0000 7fe3c5151640 1 --2- 172.21.15.5:0/2573207330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3c006d2d0 0x7fe3c0142140 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:39.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.229+0000 7fe3c5151640 1 -- 172.21.15.5:0/2573207330 >> 172.21.15.5:0/2573207330 conn(0x7fe3c00b5670 msgr2=0x7fe3c013e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:39.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.229+0000 7fe3c5151640 1 -- 172.21.15.5:0/2573207330 shutdown_connections 2024-03-18T15:33:39.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.229+0000 7fe3c5151640 1 -- 172.21.15.5:0/2573207330 wait complete. 2024-03-18T15:33:39.248 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:33:39.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.369+0000 7ff9a00c0640 1 Processor -- start 2024-03-18T15:33:39.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.369+0000 7ff9a00c0640 1 -- start start 2024-03-18T15:33:39.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.369+0000 7ff9a00c0640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff998141b20 0x7ff99806d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:39.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.369+0000 7ff9a00c0640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9980b7d80 con 0x7ff9980b1bc0 2024-03-18T15:33:39.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.369+0000 7ff9a00c0640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9980b6250 con 0x7ff9980b79a0 2024-03-18T15:33:39.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.369+0000 7ff9a00c0640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7ff9980b6530 con 0x7ff998141b20 2024-03-18T15:33:39.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.369+0000 7ff99e65d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff998141b20 0x7ff99806d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:39.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.369+0000 7ff99e65d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff998141b20 0x7ff99806d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36034/0 (socket says 172.21.15.5:36034) 2024-03-18T15:33:39.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.369+0000 7ff99d65b640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff9980b1bc0 0x7ff99814fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:39370/0 (socket says 172.21.15.5:39370) 2024-03-18T15:33:39.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.369+0000 7ff99e65d640 1 -- 172.21.15.5:0/482386319 learned_addr learned my addr 172.21.15.5:0/482386319 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:39.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.369+0000 7ff99e65d640 1 -- 172.21.15.5:0/482386319 >> v1:172.21.15.80:6789/0 conn(0x7ff9980b79a0 legacy=0x7ff998141410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:39.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.369+0000 7ff99e65d640 1 -- 172.21.15.5:0/482386319 >> v1:172.21.15.5:6789/0 conn(0x7ff9980b1bc0 legacy=0x7ff99814fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:39.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.369+0000 7ff99e65d640 1 -- 172.21.15.5:0/482386319 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9980b56b0 con 0x7ff998141b20 2024-03-18T15:33:39.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.373+0000 7ff99e65d640 1 --2- 172.21.15.5:0/482386319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff998141b20 0x7ff99806d310 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff990009870 tx=0x7ff99001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c01182837adc185e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:39.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.373+0000 7ff99ce5a640 1 -- 172.21.15.5:0/482386319 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9900ab430 con 0x7ff998141b20 2024-03-18T15:33:39.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.373+0000 7ff99ce5a640 1 -- 172.21.15.5:0/482386319 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9900b0070 con 0x7ff998141b20 2024-03-18T15:33:39.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.373+0000 7ff99ce5a640 1 -- 172.21.15.5:0/482386319 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9900ab7f0 con 0x7ff998141b20 2024-03-18T15:33:39.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.373+0000 7ff9a00c0640 1 -- 172.21.15.5:0/482386319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff998141b20 msgr2=0x7ff99806d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:39.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.373+0000 7ff9a00c0640 1 --2- 172.21.15.5:0/482386319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff998141b20 0x7ff99806d310 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff990009870 tx=0x7ff99001c520 comp rx=0 tx=0).stop 2024-03-18T15:33:39.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.373+0000 7ff9a00c0640 1 -- 172.21.15.5:0/482386319 shutdown_connections 2024-03-18T15:33:39.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.373+0000 7ff9a00c0640 1 --2- 172.21.15.5:0/482386319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff998141b20 0x7ff99806d310 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:39.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.373+0000 7ff9a00c0640 1 -- 172.21.15.5:0/482386319 >> 172.21.15.5:0/482386319 conn(0x7ff998066fc0 msgr2=0x7ff9980673d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:39.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.373+0000 7ff9a00c0640 1 -- 172.21.15.5:0/482386319 shutdown_connections 2024-03-18T15:33:39.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.373+0000 7ff9a00c0640 1 -- 172.21.15.5:0/482386319 wait complete. 2024-03-18T15:33:39.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.373+0000 7ff9a00c0640 1 Processor -- start 2024-03-18T15:33:39.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.373+0000 7ff9a00c0640 1 -- start start 2024-03-18T15:33:39.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.373+0000 7ff9a00c0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9980b79a0 0x7ff998144640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:39.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.373+0000 7ff9a00c0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff998141b20 0x7ff9981339a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:39.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.373+0000 7ff9a00c0640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff9981e3b80 0x7ff9981e0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:39.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.373+0000 7ff9a00c0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff998155fd0 con 0x7ff9980b79a0 2024-03-18T15:33:39.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.373+0000 7ff9a00c0640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff998072cf0 con 0x7ff998141b20 2024-03-18T15:33:39.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.373+0000 7ff9a00c0640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7ff998156150 con 0x7ff9981e3b80 2024-03-18T15:33:39.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.373+0000 7ff99de5c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9980b79a0 0x7ff998144640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:39.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.373+0000 7ff99de5c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9980b79a0 0x7ff998144640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36042/0 (socket says 172.21.15.5:36042) 2024-03-18T15:33:39.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.373+0000 7ff99de5c640 1 -- 172.21.15.5:0/359922328 learned_addr learned my addr 172.21.15.5:0/359922328 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:39.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.373+0000 7ff99d65b640 1 --2- 172.21.15.5:0/359922328 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff998141b20 0x7ff9981339a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:39.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.373+0000 7ff99de5c640 1 -- 172.21.15.5:0/359922328 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff9981e3b80 msgr2=0x7ff9981e0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:39.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.373+0000 7ff99de5c640 1 --2- 172.21.15.5:0/359922328 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff9981e3b80 0x7ff9981e0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:39.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.373+0000 7ff99de5c640 1 -- 172.21.15.5:0/359922328 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff998141b20 msgr2=0x7ff9981339a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:39.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.373+0000 7ff99de5c640 1 --2- 172.21.15.5:0/359922328 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff998141b20 0x7ff9981339a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:39.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.373+0000 7ff99de5c640 1 -- 172.21.15.5:0/359922328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff998139cb0 con 0x7ff9980b79a0 2024-03-18T15:33:39.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.373+0000 7ff99de5c640 1 --2- 172.21.15.5:0/359922328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9980b79a0 0x7ff998144640 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff99400c480 tx=0x7ff99400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:39.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.373+0000 7ff98a7fc640 1 -- 172.21.15.5:0/359922328 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff994012830 con 0x7ff9980b79a0 2024-03-18T15:33:39.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.373+0000 7ff98a7fc640 1 -- 172.21.15.5:0/359922328 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff994017070 con 0x7ff9980b79a0 2024-03-18T15:33:39.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.373+0000 7ff9a00c0640 1 -- 172.21.15.5:0/359922328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff998142330 con 0x7ff9980b79a0 2024-03-18T15:33:39.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.373+0000 7ff98a7fc640 1 -- 172.21.15.5:0/359922328 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff994012c30 con 0x7ff9980b79a0 2024-03-18T15:33:39.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.373+0000 7ff9a00c0640 1 -- 172.21.15.5:0/359922328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9981e43b0 con 0x7ff9980b79a0 2024-03-18T15:33:39.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.377+0000 7ff9a00c0640 1 -- 172.21.15.5:0/359922328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff99804f550 con 0x7ff9980b79a0 2024-03-18T15:33:39.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.377+0000 7ff98a7fc640 1 -- 172.21.15.5:0/359922328 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7ff99401c460 con 0x7ff9980b79a0 2024-03-18T15:33:39.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.377+0000 7ff98a7fc640 1 --2- 172.21.15.5:0/359922328 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7ff97c067660 0x7ff97c069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:39.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.377+0000 7ff98a7fc640 1 -- 172.21.15.5:0/359922328 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7ff994085300 con 0x7ff9980b79a0 2024-03-18T15:33:39.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.377+0000 7ff99d65b640 1 --2- 172.21.15.5:0/359922328 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7ff97c067660 0x7ff97c069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:39.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.377+0000 7ff99d65b640 1 --2- 172.21.15.5:0/359922328 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7ff97c067660 0x7ff97c069b20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff98c007fd0 tx=0x7ff98c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:39.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.381+0000 7ff98a7fc640 1 -- 172.21.15.5:0/359922328 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7ff994056360 con 0x7ff9980b79a0 2024-03-18T15:33:39.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.541+0000 7ff9a00c0640 1 -- 172.21.15.5:0/359922328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff9980723d0 con 0x7ff9980b79a0 2024-03-18T15:33:39.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.545+0000 7ff98a7fc640 1 -- 172.21.15.5:0/359922328 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12728 (secure 0 0 0) 0x7ff994055c60 con 0x7ff9980b79a0 2024-03-18T15:33:39.546 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:33:39.546 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":17,"fsid":"ef54fe03-5aa7-4eb2-9b1e-66c9cc345b94","created":"2024-03-18T15:33:18.086614+0000","modified":"2024-03-18T15:33:31.235982+0000","last_up_change":"2024-03-18T15:33:29.229911+0000","last_in_change":"2024-03-18T15:33:20.323840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:28.963186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df1c754b-af59-4d16-8919-fbb6386b3c48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6804","nonce":976264754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6808","nonce":976264754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6815","nonce":976264754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6811","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6813","nonce":976264754}]},"public_addr":"172.21.15.5:6804/976264754","cluster_addr":"172.21.15.5:6808/976264754","heartbeat_back_addr":"172.21.15.5:6815/976264754","heartbeat_front_addr":"172.21.15.5:6813/976264754","state":["exists","up"]},{"osd":1,"uuid":"b9a08b5e-8af4-4144-bdba-d69ed1ac627a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6801","nonce":3901761126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6803","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6805","nonce":3901761126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6812","nonce":3901761126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6807","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6809","nonce":3901761126}]},"public_addr":"172.21.15.5:6801/3901761126","cluster_addr":"172.21.15.5:6805/3901761126","heartbeat_back_addr":"172.21.15.5:6812/3901761126","heartbeat_front_addr":"172.21.15.5:6809/3901761126","state":["exists","up"]},{"osd":2,"uuid":"c3e8bb76-e382-439d-b380-62ef92f1255a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6817","nonce":502968448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6819","nonce":502968448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6823","nonce":502968448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6821","nonce":502968448}]},"public_addr":"172.21.15.5:6817/502968448","cluster_addr":"172.21.15.5:6819/502968448","heartbeat_back_addr":"172.21.15.5:6823/502968448","heartbeat_front_addr":"172.21.15.5:6821/502968448","state":["exists","up"]},{"osd":3,"uuid":"e8c94625-f2cd-4eba-b53d-2e7356e1c02a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6825","nonce":2425932735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6827","nonce":2425932735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6831","nonce":2425932735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6829","nonce":2425932735}]},"public_addr":"172.21.15.5:6825/2425932735","cluster_addr":"172.21.15.5:6827/2425932735","heartbeat_back_addr":"172.21.15.5:6831/2425932735","heartbeat_front_addr":"172.21.15.5:6829/2425932735","state":["exists","up"]},{"osd":4,"uuid":"0099dd52-6265-4d32-82e4-3ef16e73add0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6817","nonce":976514285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6819","nonce":976514285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6823","nonce":976514285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6821","nonce":976514285}]},"public_addr":"172.21.15.80:6817/976514285","cluster_addr":"172.21.15.80:6819/976514285","heartbeat_back_addr":"172.21.15.80:6823/976514285","heartbeat_front_addr":"172.21.15.80:6821/976514285","state":["exists","up"]},{"osd":5,"uuid":"ed10c84f-8c39-41cc-9b64-7c482a94a48e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6825","nonce":627153447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6827","nonce":627153447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6831","nonce":627153447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6829","nonce":627153447}]},"public_addr":"172.21.15.80:6825/627153447","cluster_addr":"172.21.15.80:6827/627153447","heartbeat_back_addr":"172.21.15.80:6831/627153447","heartbeat_front_addr":"172.21.15.80:6829/627153447","state":["exists","up"]},{"osd":6,"uuid":"c7a3538a-7048-4fba-8781-b440f03c25db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6803","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6805","nonce":488342527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6807","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6809","nonce":488342527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6815","nonce":488342527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6811","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6813","nonce":488342527}]},"public_addr":"172.21.15.80:6805/488342527","cluster_addr":"172.21.15.80:6809/488342527","heartbeat_back_addr":"172.21.15.80:6815/488342527","heartbeat_front_addr":"172.21.15.80:6813/488342527","state":["exists","up"]},{"osd":7,"uuid":"f9418104-3a97-4110-bc3a-1f8151d0f757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6801","nonce":982831666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6804","nonce":982831666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6812","nonce":982831666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6808","nonce":982831666}]},"public_addr":"172.21.15.80:6801/982831666","cluster_addr":"172.21.15.80:6804/982831666","heartbeat_back_addr":"172.21.15.80:6812/982831666","heartbeat_front_addr":"172.21.15.80:6808/982831666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.970828+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.858116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.987556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.864603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.756293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.591502+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:33:39.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.545+0000 7ff9a00c0640 1 -- 172.21.15.5:0/359922328 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7ff97c067660 msgr2=0x7ff97c069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:39.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.545+0000 7ff9a00c0640 1 --2- 172.21.15.5:0/359922328 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7ff97c067660 0x7ff97c069b20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff98c007fd0 tx=0x7ff98c009040 comp rx=0 tx=0).stop 2024-03-18T15:33:39.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.545+0000 7ff9a00c0640 1 -- 172.21.15.5:0/359922328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9980b79a0 msgr2=0x7ff998144640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:39.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.545+0000 7ff9a00c0640 1 --2- 172.21.15.5:0/359922328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9980b79a0 0x7ff998144640 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff99400c480 tx=0x7ff99400c940 comp rx=0 tx=0).stop 2024-03-18T15:33:39.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.545+0000 7ff9a00c0640 1 -- 172.21.15.5:0/359922328 shutdown_connections 2024-03-18T15:33:39.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.545+0000 7ff9a00c0640 1 --2- 172.21.15.5:0/359922328 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7ff97c067660 0x7ff97c069b20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:39.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.545+0000 7ff9a00c0640 1 --2- 172.21.15.5:0/359922328 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff9981e3b80 0x7ff9981e0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:39.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.545+0000 7ff9a00c0640 1 --2- 172.21.15.5:0/359922328 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff998141b20 0x7ff9981339a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:39.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.545+0000 7ff9a00c0640 1 --2- 172.21.15.5:0/359922328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9980b79a0 0x7ff998144640 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:39.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.545+0000 7ff9a00c0640 1 -- 172.21.15.5:0/359922328 >> 172.21.15.5:0/359922328 conn(0x7ff998066fc0 msgr2=0x7ff9980bac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:39.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.545+0000 7ff9a00c0640 1 -- 172.21.15.5:0/359922328 shutdown_connections 2024-03-18T15:33:39.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.545+0000 7ff9a00c0640 1 -- 172.21.15.5:0/359922328 wait complete. 2024-03-18T15:33:39.563 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T15:33:28.963186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T15:33:39.564 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T15:33:39.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.689+0000 7f0a541f5640 1 Processor -- start 2024-03-18T15:33:39.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.689+0000 7f0a541f5640 1 -- start start 2024-03-18T15:33:39.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.689+0000 7f0a541f5640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0a4c06d2d0 0x7f0a4c0b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:39.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.689+0000 7f0a541f5640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a4c13f260 0x7f0a4c13f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:39.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.689+0000 7f0a541f5640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a4c0b0ea0 con 0x7f0a4c1453a0 2024-03-18T15:33:39.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.689+0000 7f0a541f5640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f0a4c0728c0 con 0x7f0a4c06d2d0 2024-03-18T15:33:39.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.689+0000 7f0a541f5640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f0a4c06d6b0 con 0x7f0a4c13f260 2024-03-18T15:33:39.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.693+0000 7f0a52792640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f0a4c1453a0 0x7f0a4c145810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:39380/0 (socket says 172.21.15.5:39380) 2024-03-18T15:33:39.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.693+0000 7f0a52792640 1 -- 172.21.15.5:0/1047196457 learned_addr learned my addr 172.21.15.5:0/1047196457 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:33:39.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.693+0000 7f0a51f91640 1 --2- 172.21.15.5:0/1047196457 >> v2:172.21.15.80:3300/0 conn(0x7f0a4c06d2d0 0x7f0a4c0b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:39.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.693+0000 7f0a51790640 1 --2- 172.21.15.5:0/1047196457 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a4c13f260 0x7f0a4c13f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:39.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.693+0000 7f0a50f8f640 1 -- 172.21.15.5:0/1047196457 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2033053421 0 0) 0x7f0a4c0b0ea0 con 0x7f0a4c1453a0 2024-03-18T15:33:39.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.693+0000 7f0a50f8f640 1 -- 172.21.15.5:0/1047196457 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0a38003530 con 0x7f0a4c1453a0 2024-03-18T15:33:39.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.693+0000 7f0a50f8f640 1 -- 172.21.15.5:0/1047196457 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4122359429 0 0) 0x7f0a38003530 con 0x7f0a4c1453a0 2024-03-18T15:33:39.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.693+0000 7f0a51f91640 1 -- 172.21.15.5:0/1047196457 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a4c13f260 msgr2=0x7f0a4c13f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:39.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.693+0000 7f0a51f91640 1 --2- 172.21.15.5:0/1047196457 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a4c13f260 0x7f0a4c13f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:39.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.693+0000 7f0a51f91640 1 -- 172.21.15.5:0/1047196457 >> v1:172.21.15.5:6789/0 conn(0x7f0a4c1453a0 legacy=0x7f0a4c145810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:39.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.693+0000 7f0a51f91640 1 -- 172.21.15.5:0/1047196457 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a4c0b0bc0 con 0x7f0a4c06d2d0 2024-03-18T15:33:39.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.693+0000 7f0a51f91640 1 --2- 172.21.15.5:0/1047196457 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a4c06d2d0 0x7f0a4c0b23a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0a48009d90 tx=0x7f0a4801c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92f73cfc7faf38c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:39.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.693+0000 7f0a50f8f640 1 -- 172.21.15.5:0/1047196457 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a480b2070 con 0x7f0a4c06d2d0 2024-03-18T15:33:39.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.693+0000 7f0a50f8f640 1 -- 172.21.15.5:0/1047196457 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a480a5510 con 0x7f0a4c06d2d0 2024-03-18T15:33:39.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.693+0000 7f0a50f8f640 1 -- 172.21.15.5:0/1047196457 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a44001680 con 0x7f0a4c06d2d0 2024-03-18T15:33:39.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.697+0000 7f0a541f5640 1 -- 172.21.15.5:0/1047196457 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a4c06d2d0 msgr2=0x7f0a4c0b23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:39.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.697+0000 7f0a541f5640 1 --2- 172.21.15.5:0/1047196457 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a4c06d2d0 0x7f0a4c0b23a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0a48009d90 tx=0x7f0a4801c800 comp rx=0 tx=0).stop 2024-03-18T15:33:39.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.697+0000 7f0a541f5640 1 -- 172.21.15.5:0/1047196457 shutdown_connections 2024-03-18T15:33:39.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.697+0000 7f0a541f5640 1 --2- 172.21.15.5:0/1047196457 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a4c13f260 0x7f0a4c13f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:39.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.697+0000 7f0a541f5640 1 --2- 172.21.15.5:0/1047196457 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a4c06d2d0 0x7f0a4c0b23a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:39.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.697+0000 7f0a541f5640 1 -- 172.21.15.5:0/1047196457 >> 172.21.15.5:0/1047196457 conn(0x7f0a4c0b5670 msgr2=0x7f0a4c0b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:39.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.697+0000 7f0a541f5640 1 -- 172.21.15.5:0/1047196457 shutdown_connections 2024-03-18T15:33:39.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.697+0000 7f0a541f5640 1 -- 172.21.15.5:0/1047196457 wait complete. 2024-03-18T15:33:39.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.697+0000 7f0a541f5640 1 Processor -- start 2024-03-18T15:33:39.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.697+0000 7f0a541f5640 1 -- start start 2024-03-18T15:33:39.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.697+0000 7f0a541f5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a4c06d2d0 0x7f0a4c1e3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:39.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.697+0000 7f0a541f5640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a4c13f260 0x7f0a4c0b7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:39.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.697+0000 7f0a541f5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a4c1453a0 0x7f0a4c0adab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:39.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.697+0000 7f0a541f5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4c072ed0 con 0x7f0a4c1453a0 2024-03-18T15:33:39.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.697+0000 7f0a541f5640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0a4c072d50 con 0x7f0a4c06d2d0 2024-03-18T15:33:39.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.697+0000 7f0a541f5640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f0a4c0b5140 con 0x7f0a4c13f260 2024-03-18T15:33:39.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.697+0000 7f0a52792640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a4c1453a0 0x7f0a4c0adab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:39.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.697+0000 7f0a52792640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a4c1453a0 0x7f0a4c0adab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36052/0 (socket says 172.21.15.5:36052) 2024-03-18T15:33:39.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.697+0000 7f0a52792640 1 -- 172.21.15.5:0/4254860284 learned_addr learned my addr 172.21.15.5:0/4254860284 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:39.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.697+0000 7f0a51f91640 1 --2- 172.21.15.5:0/4254860284 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a4c06d2d0 0x7f0a4c1e3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:39.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.697+0000 7f0a51790640 1 --2- 172.21.15.5:0/4254860284 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a4c13f260 0x7f0a4c0b7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:39.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.697+0000 7f0a52792640 1 -- 172.21.15.5:0/4254860284 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a4c13f260 msgr2=0x7f0a4c0b7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:39.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.697+0000 7f0a52792640 1 --2- 172.21.15.5:0/4254860284 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a4c13f260 0x7f0a4c0b7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:39.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.697+0000 7f0a52792640 1 -- 172.21.15.5:0/4254860284 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a4c06d2d0 msgr2=0x7f0a4c1e3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:39.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.697+0000 7f0a52792640 1 --2- 172.21.15.5:0/4254860284 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a4c06d2d0 0x7f0a4c1e3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:39.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.697+0000 7f0a52792640 1 -- 172.21.15.5:0/4254860284 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a4c0afad0 con 0x7f0a4c1453a0 2024-03-18T15:33:39.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.697+0000 7f0a52792640 1 --2- 172.21.15.5:0/4254860284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a4c1453a0 0x7f0a4c0adab0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0a4400c750 tx=0x7f0a4400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:39.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.697+0000 7f0a427fc640 1 -- 172.21.15.5:0/4254860284 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a44005ae0 con 0x7f0a4c1453a0 2024-03-18T15:33:39.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.697+0000 7f0a427fc640 1 -- 172.21.15.5:0/4254860284 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a44005ca0 con 0x7f0a4c1453a0 2024-03-18T15:33:39.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.697+0000 7f0a427fc640 1 -- 172.21.15.5:0/4254860284 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a440072b0 con 0x7f0a4c1453a0 2024-03-18T15:33:39.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.697+0000 7f0a541f5640 1 -- 172.21.15.5:0/4254860284 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a4c1e3620 con 0x7f0a4c1453a0 2024-03-18T15:33:39.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.697+0000 7f0a541f5640 1 -- 172.21.15.5:0/4254860284 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a4c13e750 con 0x7f0a4c1453a0 2024-03-18T15:33:39.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.701+0000 7f0a541f5640 1 -- 172.21.15.5:0/4254860284 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a4c04f550 con 0x7f0a4c1453a0 2024-03-18T15:33:39.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.701+0000 7f0a427fc640 1 -- 172.21.15.5:0/4254860284 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f0a44007450 con 0x7f0a4c1453a0 2024-03-18T15:33:39.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.701+0000 7f0a427fc640 1 --2- 172.21.15.5:0/4254860284 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0a30067660 0x7f0a30069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:39.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.701+0000 7f0a427fc640 1 -- 172.21.15.5:0/4254860284 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f0a44085380 con 0x7f0a4c1453a0 2024-03-18T15:33:39.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.701+0000 7f0a51f91640 1 --2- 172.21.15.5:0/4254860284 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0a30067660 0x7f0a30069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:39.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.701+0000 7f0a51f91640 1 --2- 172.21.15.5:0/4254860284 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0a30067660 0x7f0a30069b20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0a480af000 tx=0x7f0a480a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:39.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.705+0000 7f0a427fc640 1 -- 172.21.15.5:0/4254860284 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f0a44010300 con 0x7f0a4c1453a0 2024-03-18T15:33:39.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.865+0000 7f0a541f5640 1 -- 172.21.15.5:0/4254860284 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0a4c143490 con 0x7f0a4c1453a0 2024-03-18T15:33:39.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.865+0000 7f0a427fc640 1 -- 172.21.15.5:0/4254860284 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) ==== 93+0+10 (secure 0 0 0) 0x7f0a44056340 con 0x7f0a4c1453a0 2024-03-18T15:33:39.869 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-03-18T15:33:39.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.869+0000 7f0a541f5640 1 -- 172.21.15.5:0/4254860284 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0a30067660 msgr2=0x7f0a30069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:39.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.869+0000 7f0a541f5640 1 --2- 172.21.15.5:0/4254860284 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0a30067660 0x7f0a30069b20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0a480af000 tx=0x7f0a480a7070 comp rx=0 tx=0).stop 2024-03-18T15:33:39.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.869+0000 7f0a541f5640 1 -- 172.21.15.5:0/4254860284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a4c1453a0 msgr2=0x7f0a4c0adab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:39.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.869+0000 7f0a541f5640 1 --2- 172.21.15.5:0/4254860284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a4c1453a0 0x7f0a4c0adab0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0a4400c750 tx=0x7f0a4400cc10 comp rx=0 tx=0).stop 2024-03-18T15:33:39.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.869+0000 7f0a541f5640 1 -- 172.21.15.5:0/4254860284 shutdown_connections 2024-03-18T15:33:39.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.869+0000 7f0a541f5640 1 --2- 172.21.15.5:0/4254860284 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0a30067660 0x7f0a30069b20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:39.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.869+0000 7f0a541f5640 1 --2- 172.21.15.5:0/4254860284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a4c1453a0 0x7f0a4c0adab0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:39.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.869+0000 7f0a541f5640 1 --2- 172.21.15.5:0/4254860284 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a4c13f260 0x7f0a4c0b7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:39.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.869+0000 7f0a541f5640 1 --2- 172.21.15.5:0/4254860284 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a4c06d2d0 0x7f0a4c1e3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:39.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.869+0000 7f0a541f5640 1 -- 172.21.15.5:0/4254860284 >> 172.21.15.5:0/4254860284 conn(0x7f0a4c0b5670 msgr2=0x7f0a4c13bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:39.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.869+0000 7f0a541f5640 1 -- 172.21.15.5:0/4254860284 shutdown_connections 2024-03-18T15:33:39.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:39.869+0000 7f0a541f5640 1 -- 172.21.15.5:0/4254860284 wait complete. 2024-03-18T15:33:39.888 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-18T15:33:40.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.009+0000 7f26fe39e640 1 Processor -- start 2024-03-18T15:33:40.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.009+0000 7f26fe39e640 1 -- start start 2024-03-18T15:33:40.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.013+0000 7f26fe39e640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f26f80af790 0x7f26f8072b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:40.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.013+0000 7f26fe39e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f26f806d070 0x7f26f8147780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:40.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.013+0000 7f26fe39e640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26f80afb70 con 0x7f26f80b26a0 2024-03-18T15:33:40.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.013+0000 7f26fe39e640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f26f80b4a20 con 0x7f26f806d070 2024-03-18T15:33:40.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.013+0000 7f26fe39e640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f26f80b4d00 con 0x7f26f80af790 2024-03-18T15:33:40.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.013+0000 7f26f77fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f26f806d070 0x7f26f8147780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:40.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.013+0000 7f26f77fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f26f806d070 0x7f26f8147780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36066/0 (socket says 172.21.15.5:36066) 2024-03-18T15:33:40.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.013+0000 7f26f77fe640 1 -- 172.21.15.5:0/2398429341 learned_addr learned my addr 172.21.15.5:0/2398429341 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:40.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.013+0000 7f26f7fff640 1 --2- 172.21.15.5:0/2398429341 >> v2:172.21.15.80:3300/0 conn(0x7f26f80af790 0x7f26f8072b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:40.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.013+0000 7f26f77fe640 1 -- 172.21.15.5:0/2398429341 >> v2:172.21.15.80:3300/0 conn(0x7f26f80af790 msgr2=0x7f26f8072b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:40.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.013+0000 7f26f77fe640 1 --2- 172.21.15.5:0/2398429341 >> v2:172.21.15.80:3300/0 conn(0x7f26f80af790 0x7f26f8072b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:40.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.013+0000 7f26f77fe640 1 -- 172.21.15.5:0/2398429341 >> v1:172.21.15.80:6789/0 conn(0x7f26f80b26a0 legacy=0x7f26f80b2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:40.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.013+0000 7f26f77fe640 1 -- 172.21.15.5:0/2398429341 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26f806d4b0 con 0x7f26f806d070 2024-03-18T15:33:40.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.013+0000 7f26f77fe640 1 --2- 172.21.15.5:0/2398429341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26f806d070 0x7f26f8147780 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f26f806dd00 tx=0x7f26e801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7fef8e5c914c079 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:40.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.013+0000 7f26f6ffd640 1 -- 172.21.15.5:0/2398429341 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e80a5630 con 0x7f26f806d070 2024-03-18T15:33:40.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.013+0000 7f26f6ffd640 1 -- 172.21.15.5:0/2398429341 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26e80a57d0 con 0x7f26f806d070 2024-03-18T15:33:40.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.013+0000 7f26f6ffd640 1 -- 172.21.15.5:0/2398429341 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e80a5ba0 con 0x7f26f806d070 2024-03-18T15:33:40.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.017+0000 7f26fe39e640 1 -- 172.21.15.5:0/2398429341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26f806d070 msgr2=0x7f26f8147780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:40.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.017+0000 7f26fe39e640 1 --2- 172.21.15.5:0/2398429341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26f806d070 0x7f26f8147780 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f26f806dd00 tx=0x7f26e801c960 comp rx=0 tx=0).stop 2024-03-18T15:33:40.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.017+0000 7f26fe39e640 1 -- 172.21.15.5:0/2398429341 shutdown_connections 2024-03-18T15:33:40.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.017+0000 7f26fe39e640 1 --2- 172.21.15.5:0/2398429341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26f806d070 0x7f26f8147780 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:40.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.017+0000 7f26fe39e640 1 --2- 172.21.15.5:0/2398429341 >> v2:172.21.15.80:3300/0 conn(0x7f26f80af790 0x7f26f8072b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:40.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.017+0000 7f26fe39e640 1 -- 172.21.15.5:0/2398429341 >> 172.21.15.5:0/2398429341 conn(0x7f26f80b5670 msgr2=0x7f26f80b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:40.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.017+0000 7f26fe39e640 1 -- 172.21.15.5:0/2398429341 shutdown_connections 2024-03-18T15:33:40.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.017+0000 7f26fe39e640 1 -- 172.21.15.5:0/2398429341 wait complete. 2024-03-18T15:33:40.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.017+0000 7f26fe39e640 1 Processor -- start 2024-03-18T15:33:40.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.017+0000 7f26fe39e640 1 -- start start 2024-03-18T15:33:40.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.017+0000 7f26fe39e640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f26f806d070 0x7f26f8072000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.017+0000 7f26fe39e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26f80af790 0x7f26f81e1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.017+0000 7f26fe39e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26f80b26a0 0x7f26f81ddf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.017+0000 7f26fe39e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f26f814f6d0 con 0x7f26f80af790 2024-03-18T15:33:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.017+0000 7f26fe39e640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f26f814f550 con 0x7f26f80b26a0 2024-03-18T15:33:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.017+0000 7f26fe39e640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f26f814f850 con 0x7f26f806d070 2024-03-18T15:33:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.017+0000 7f26f77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26f80af790 0x7f26f81e1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.017+0000 7f26f77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26f80af790 0x7f26f81e1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36082/0 (socket says 172.21.15.5:36082) 2024-03-18T15:33:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.017+0000 7f26f77fe640 1 -- 172.21.15.5:0/3032833176 learned_addr learned my addr 172.21.15.5:0/3032833176 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.017+0000 7f26fc93b640 1 --2- 172.21.15.5:0/3032833176 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26f80b26a0 0x7f26f81ddf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.017+0000 7f26f77fe640 1 -- 172.21.15.5:0/3032833176 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f26f806d070 msgr2=0x7f26f8072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.017+0000 7f26f7fff640 1 --2- 172.21.15.5:0/3032833176 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f26f806d070 0x7f26f8072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.017+0000 7f26f77fe640 1 --2- 172.21.15.5:0/3032833176 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f26f806d070 0x7f26f8072000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.017+0000 7f26f77fe640 1 -- 172.21.15.5:0/3032833176 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26f80b26a0 msgr2=0x7f26f81ddf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:40.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.017+0000 7f26f77fe640 1 --2- 172.21.15.5:0/3032833176 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26f80b26a0 0x7f26f81ddf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:40.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.017+0000 7f26f77fe640 1 -- 172.21.15.5:0/3032833176 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26f8071b70 con 0x7f26f80af790 2024-03-18T15:33:40.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.017+0000 7f26fc93b640 1 --2- 172.21.15.5:0/3032833176 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26f80b26a0 0x7f26f81ddf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:33:40.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.017+0000 7f26f77fe640 1 --2- 172.21.15.5:0/3032833176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26f80af790 0x7f26f81e1720 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f26e8019570 tx=0x7f26e8019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:40.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.017+0000 7f26f4ff9640 1 -- 172.21.15.5:0/3032833176 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e80bc020 con 0x7f26f80af790 2024-03-18T15:33:40.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.017+0000 7f26f4ff9640 1 -- 172.21.15.5:0/3032833176 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26e80b8670 con 0x7f26f80af790 2024-03-18T15:33:40.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.017+0000 7f26f4ff9640 1 -- 172.21.15.5:0/3032833176 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e8009510 con 0x7f26f80af790 2024-03-18T15:33:40.022 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.017+0000 7f26fe39e640 1 -- 172.21.15.5:0/3032833176 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26f8070670 con 0x7f26f80af790 2024-03-18T15:33:40.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.017+0000 7f26fe39e640 1 -- 172.21.15.5:0/3032833176 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26f8071890 con 0x7f26f80af790 2024-03-18T15:33:40.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.017+0000 7f26fe39e640 1 -- 172.21.15.5:0/3032833176 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26f8070b90 con 0x7f26f80af790 2024-03-18T15:33:40.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.021+0000 7f26f4ff9640 1 -- 172.21.15.5:0/3032833176 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f26e80bf020 con 0x7f26f80af790 2024-03-18T15:33:40.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.021+0000 7f26f4ff9640 1 --2- 172.21.15.5:0/3032833176 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f26d8067690 0x7f26d8069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:40.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.021+0000 7f26f4ff9640 1 -- 172.21.15.5:0/3032833176 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f26e8120470 con 0x7f26f80af790 2024-03-18T15:33:40.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.021+0000 7f26f7fff640 1 --2- 172.21.15.5:0/3032833176 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f26d8067690 0x7f26d8069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:40.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.021+0000 7f26f7fff640 1 --2- 172.21.15.5:0/3032833176 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f26d8067690 0x7f26d8069b50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f26f813e1e0 tx=0x7f26e0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:40.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.025+0000 7f26f4ff9640 1 -- 172.21.15.5:0/3032833176 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f26e80c6030 con 0x7f26f80af790 2024-03-18T15:33:40.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.181+0000 7f26fe39e640 1 -- 172.21.15.5:0/3032833176 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f26f81438f0 con 0x7f26f80af790 2024-03-18T15:33:40.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.181+0000 7f26f4ff9640 1 -- 172.21.15.5:0/3032833176 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) ==== 84+0+65 (secure 0 0 0) 0x7f26e80165e0 con 0x7f26f80af790 2024-03-18T15:33:40.184 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:33:40.184 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2024-03-18T15:33:40.184 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-03-18T15:33:40.184 INFO:teuthology.orchestra.run.smithi005.stdout: "poolnum": 1, 2024-03-18T15:33:40.184 INFO:teuthology.orchestra.run.smithi005.stdout: "poolname": ".mgr" 2024-03-18T15:33:40.184 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-03-18T15:33:40.184 INFO:teuthology.orchestra.run.smithi005.stdout:] 2024-03-18T15:33:40.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.185+0000 7f26fe39e640 1 -- 172.21.15.5:0/3032833176 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f26d8067690 msgr2=0x7f26d8069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:40.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.185+0000 7f26fe39e640 1 --2- 172.21.15.5:0/3032833176 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f26d8067690 0x7f26d8069b50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f26f813e1e0 tx=0x7f26e0008040 comp rx=0 tx=0).stop 2024-03-18T15:33:40.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.185+0000 7f26fe39e640 1 -- 172.21.15.5:0/3032833176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26f80af790 msgr2=0x7f26f81e1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:40.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.185+0000 7f26fe39e640 1 --2- 172.21.15.5:0/3032833176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26f80af790 0x7f26f81e1720 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f26e8019570 tx=0x7f26e8019700 comp rx=0 tx=0).stop 2024-03-18T15:33:40.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.185+0000 7f26fe39e640 1 -- 172.21.15.5:0/3032833176 shutdown_connections 2024-03-18T15:33:40.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.185+0000 7f26fe39e640 1 --2- 172.21.15.5:0/3032833176 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f26d8067690 0x7f26d8069b50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:40.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.185+0000 7f26fe39e640 1 --2- 172.21.15.5:0/3032833176 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26f80b26a0 0x7f26f81ddf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:40.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.185+0000 7f26fe39e640 1 --2- 172.21.15.5:0/3032833176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26f80af790 0x7f26f81e1720 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:40.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.185+0000 7f26fe39e640 1 --2- 172.21.15.5:0/3032833176 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f26f806d070 0x7f26f8072000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:40.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.185+0000 7f26fe39e640 1 -- 172.21.15.5:0/3032833176 >> 172.21.15.5:0/3032833176 conn(0x7f26f80b5670 msgr2=0x7f26f80b6a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:40.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.185+0000 7f26fe39e640 1 -- 172.21.15.5:0/3032833176 shutdown_connections 2024-03-18T15:33:40.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.185+0000 7f26fe39e640 1 -- 172.21.15.5:0/3032833176 wait complete. 2024-03-18T15:33:40.202 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-18T15:33:40.202 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-18T15:33:40.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.321+0000 7fe21aa6f640 1 Processor -- start 2024-03-18T15:33:40.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.321+0000 7fe21aa6f640 1 -- start start 2024-03-18T15:33:40.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.321+0000 7fe21aa6f640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe2140b4eb0 0x7fe2140b5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:40.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.321+0000 7fe21aa6f640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe214145560 0x7fe2141459f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:40.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.321+0000 7fe21aa6f640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe214145f30 con 0x7fe2140b4ad0 2024-03-18T15:33:40.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.321+0000 7fe21aa6f640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fe2140b6dc0 con 0x7fe214145560 2024-03-18T15:33:40.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.321+0000 7fe21aa6f640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fe2140b6770 con 0x7fe2140b4eb0 2024-03-18T15:33:40.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.321+0000 7fe21880b640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe2140b4ad0 0x7fe2140b76d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:39390/0 (socket says 172.21.15.5:39390) 2024-03-18T15:33:40.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.321+0000 7fe21900c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe214145560 0x7fe2141459f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:40.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.321+0000 7fe21880b640 1 -- 172.21.15.5:0/1836020009 learned_addr learned my addr 172.21.15.5:0/1836020009 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:33:40.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.325+0000 7fe213fff640 1 --2- 172.21.15.5:0/1836020009 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe2140b4eb0 0x7fe2140b5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:40.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.325+0000 7fe21900c640 1 -- 172.21.15.5:0/1836020009 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe2140b4eb0 msgr2=0x7fe2140b5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:40.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.325+0000 7fe21900c640 1 --2- 172.21.15.5:0/1836020009 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe2140b4eb0 0x7fe2140b5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:40.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.325+0000 7fe21900c640 1 -- 172.21.15.5:0/1836020009 >> v1:172.21.15.5:6789/0 conn(0x7fe2140b4ad0 legacy=0x7fe2140b76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:40.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.325+0000 7fe21900c640 1 -- 172.21.15.5:0/1836020009 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2140b6a50 con 0x7fe214145560 2024-03-18T15:33:40.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.325+0000 7fe2137fe640 1 -- 172.21.15.5:0/1836020009 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3048852848 0 0) 0x7fe214145f30 con 0x7fe2140b4ad0 2024-03-18T15:33:40.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.325+0000 7fe21900c640 1 --2- 172.21.15.5:0/1836020009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe214145560 0x7fe2141459f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe20c0097c0 tx=0x7fe20c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=117390bb2e7a298 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:40.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.325+0000 7fe2137fe640 1 -- 172.21.15.5:0/1836020009 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20c0b0070 con 0x7fe214145560 2024-03-18T15:33:40.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.325+0000 7fe2137fe640 1 -- 172.21.15.5:0/1836020009 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe20c0ab600 con 0x7fe214145560 2024-03-18T15:33:40.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.325+0000 7fe2137fe640 1 -- 172.21.15.5:0/1836020009 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe20c0ab9d0 con 0x7fe214145560 2024-03-18T15:33:40.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.325+0000 7fe21aa6f640 1 -- 172.21.15.5:0/1836020009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe214145560 msgr2=0x7fe2141459f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:40.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.325+0000 7fe21aa6f640 1 --2- 172.21.15.5:0/1836020009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe214145560 0x7fe2141459f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe20c0097c0 tx=0x7fe20c01c410 comp rx=0 tx=0).stop 2024-03-18T15:33:40.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.325+0000 7fe21aa6f640 1 -- 172.21.15.5:0/1836020009 shutdown_connections 2024-03-18T15:33:40.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.325+0000 7fe21aa6f640 1 --2- 172.21.15.5:0/1836020009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe214145560 0x7fe2141459f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:40.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.325+0000 7fe21aa6f640 1 --2- 172.21.15.5:0/1836020009 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe2140b4eb0 0x7fe2140b5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:40.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.325+0000 7fe21aa6f640 1 -- 172.21.15.5:0/1836020009 >> 172.21.15.5:0/1836020009 conn(0x7fe214066fc0 msgr2=0x7fe2140673d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:40.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.325+0000 7fe21aa6f640 1 -- 172.21.15.5:0/1836020009 shutdown_connections 2024-03-18T15:33:40.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.325+0000 7fe21aa6f640 1 -- 172.21.15.5:0/1836020009 wait complete. 2024-03-18T15:33:40.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.329+0000 7fe21aa6f640 1 Processor -- start 2024-03-18T15:33:40.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.329+0000 7fe21aa6f640 1 -- start start 2024-03-18T15:33:40.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.329+0000 7fe21aa6f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2140b4ad0 0x7fe214146ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:40.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.329+0000 7fe21aa6f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2140b4eb0 0x7fe21413c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:40.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.329+0000 7fe21aa6f640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe214145560 0x7fe21413d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:40.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.329+0000 7fe21aa6f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe214152870 con 0x7fe2140b4eb0 2024-03-18T15:33:40.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.329+0000 7fe21aa6f640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe2141526f0 con 0x7fe2140b4ad0 2024-03-18T15:33:40.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.329+0000 7fe21aa6f640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fe2141529f0 con 0x7fe214145560 2024-03-18T15:33:40.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.329+0000 7fe213fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2140b4eb0 0x7fe21413c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:40.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.329+0000 7fe213fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2140b4eb0 0x7fe21413c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36098/0 (socket says 172.21.15.5:36098) 2024-03-18T15:33:40.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.329+0000 7fe213fff640 1 -- 172.21.15.5:0/4034044377 learned_addr learned my addr 172.21.15.5:0/4034044377 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:40.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.329+0000 7fe21880b640 1 --2- 172.21.15.5:0/4034044377 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2140b4ad0 0x7fe214146ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:40.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.329+0000 7fe213fff640 1 -- 172.21.15.5:0/4034044377 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe214145560 msgr2=0x7fe21413d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:40.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.329+0000 7fe213fff640 1 --2- 172.21.15.5:0/4034044377 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe214145560 0x7fe21413d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:40.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.329+0000 7fe213fff640 1 -- 172.21.15.5:0/4034044377 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2140b4ad0 msgr2=0x7fe214146ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:40.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.329+0000 7fe213fff640 1 --2- 172.21.15.5:0/4034044377 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2140b4ad0 0x7fe214146ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:40.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.329+0000 7fe213fff640 1 -- 172.21.15.5:0/4034044377 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe21413df80 con 0x7fe2140b4eb0 2024-03-18T15:33:40.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.329+0000 7fe213fff640 1 --2- 172.21.15.5:0/4034044377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2140b4eb0 0x7fe21413c990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe204007ef0 tx=0x7fe20400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:40.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.329+0000 7fe2117fa640 1 -- 172.21.15.5:0/4034044377 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe204014620 con 0x7fe2140b4eb0 2024-03-18T15:33:40.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.329+0000 7fe2117fa640 1 -- 172.21.15.5:0/4034044377 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe204019070 con 0x7fe2140b4eb0 2024-03-18T15:33:40.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.329+0000 7fe2117fa640 1 -- 172.21.15.5:0/4034044377 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe204014a00 con 0x7fe2140b4eb0 2024-03-18T15:33:40.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.329+0000 7fe21aa6f640 1 -- 172.21.15.5:0/4034044377 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe21413ed00 con 0x7fe2140b4eb0 2024-03-18T15:33:40.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.329+0000 7fe21aa6f640 1 -- 172.21.15.5:0/4034044377 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe214148e20 con 0x7fe2140b4eb0 2024-03-18T15:33:40.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.329+0000 7fe21aa6f640 1 -- 172.21.15.5:0/4034044377 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe21404f5c0 con 0x7fe2140b4eb0 2024-03-18T15:33:40.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.329+0000 7fe2117fa640 1 -- 172.21.15.5:0/4034044377 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fe204014ba0 con 0x7fe2140b4eb0 2024-03-18T15:33:40.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.333+0000 7fe2117fa640 1 --2- 172.21.15.5:0/4034044377 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe1f4067660 0x7fe1f4069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:40.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.333+0000 7fe2117fa640 1 -- 172.21.15.5:0/4034044377 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fe204085dc0 con 0x7fe2140b4eb0 2024-03-18T15:33:40.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.333+0000 7fe21880b640 1 --2- 172.21.15.5:0/4034044377 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe1f4067660 0x7fe1f4069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:40.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.333+0000 7fe21880b640 1 --2- 172.21.15.5:0/4034044377 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe1f4067660 0x7fe1f4069b20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc000f70 tx=0x7fe1fc007b60 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:40.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.337+0000 7fe2117fa640 1 -- 172.21.15.5:0/4034044377 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fe204056e20 con 0x7fe2140b4eb0 2024-03-18T15:33:40.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.489+0000 7fe21aa6f640 1 -- 172.21.15.5:0/4034044377 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fe2140b7120 con 0x7fe2140b4eb0 2024-03-18T15:33:40.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.949+0000 7fe2117fa640 1 -- 172.21.15.5:0/4034044377 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) ==== 138+0+0 (secure 0 0 0) 0x7fe204056720 con 0x7fe2140b4eb0 2024-03-18T15:33:40.951 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' created 2024-03-18T15:33:40.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.953+0000 7fe21aa6f640 1 -- 172.21.15.5:0/4034044377 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe1f4067660 msgr2=0x7fe1f4069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:40.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.953+0000 7fe21aa6f640 1 --2- 172.21.15.5:0/4034044377 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe1f4067660 0x7fe1f4069b20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc000f70 tx=0x7fe1fc007b60 comp rx=0 tx=0).stop 2024-03-18T15:33:40.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.953+0000 7fe21aa6f640 1 -- 172.21.15.5:0/4034044377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2140b4eb0 msgr2=0x7fe21413c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:40.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.953+0000 7fe21aa6f640 1 --2- 172.21.15.5:0/4034044377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2140b4eb0 0x7fe21413c990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe204007ef0 tx=0x7fe20400e4b0 comp rx=0 tx=0).stop 2024-03-18T15:33:40.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.953+0000 7fe21aa6f640 1 -- 172.21.15.5:0/4034044377 shutdown_connections 2024-03-18T15:33:40.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.953+0000 7fe21aa6f640 1 --2- 172.21.15.5:0/4034044377 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe1f4067660 0x7fe1f4069b20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:40.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.953+0000 7fe21aa6f640 1 --2- 172.21.15.5:0/4034044377 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe214145560 0x7fe21413d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:40.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.953+0000 7fe21aa6f640 1 --2- 172.21.15.5:0/4034044377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2140b4eb0 0x7fe21413c990 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:40.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.953+0000 7fe21aa6f640 1 --2- 172.21.15.5:0/4034044377 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2140b4ad0 0x7fe214146ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:40.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.953+0000 7fe21aa6f640 1 -- 172.21.15.5:0/4034044377 >> 172.21.15.5:0/4034044377 conn(0x7fe214066fc0 msgr2=0x7fe214071480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:40.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.953+0000 7fe21aa6f640 1 -- 172.21.15.5:0/4034044377 shutdown_connections 2024-03-18T15:33:40.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:40.953+0000 7fe21aa6f640 1 -- 172.21.15.5:0/4034044377 wait complete. 2024-03-18T15:33:40.979 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-18T15:33:41.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.105+0000 7f1c13eb0640 1 Processor -- start 2024-03-18T15:33:41.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.105+0000 7f1c13eb0640 1 -- start start 2024-03-18T15:33:41.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.109+0000 7f1c13eb0640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1c0c06d2d0 0x7f1c0c0b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:41.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.109+0000 7f1c13eb0640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1c0c13f260 0x7f1c0c13f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:41.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.109+0000 7f1c13eb0640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1c0c1453a0 0x7f1c0c145810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:41.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.109+0000 7f1c13eb0640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f1c0c0728c0 con 0x7f1c0c13f260 2024-03-18T15:33:41.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.109+0000 7f1c13eb0640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f1c0c0b0ea0 con 0x7f1c0c1453a0 2024-03-18T15:33:41.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.109+0000 7f1c13eb0640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f1c0c06d6b0 con 0x7f1c0c06d2d0 2024-03-18T15:33:41.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.109+0000 7f1c1144b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1c0c13f260 0x7f1c0c13f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:41.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.109+0000 7f1c1144b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1c0c13f260 0x7f1c0c13f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38184/0 (socket says 172.21.15.5:38184) 2024-03-18T15:33:41.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.109+0000 7f1c1144b640 1 -- 172.21.15.5:0/543779842 learned_addr learned my addr 172.21.15.5:0/543779842 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:41.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.109+0000 7f1c1244d640 1 --2- 172.21.15.5:0/543779842 >> v2:172.21.15.80:3300/0 conn(0x7f1c0c1453a0 0x7f1c0c145810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:41.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.109+0000 7f1c11c4c640 1 --2- 172.21.15.5:0/543779842 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1c0c06d2d0 0x7f1c0c0b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:41.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.109+0000 7f1c1144b640 1 -- 172.21.15.5:0/543779842 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1c0c06d2d0 msgr2=0x7f1c0c0b23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:41.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.109+0000 7f1c1144b640 1 --2- 172.21.15.5:0/543779842 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1c0c06d2d0 0x7f1c0c0b23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:41.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.109+0000 7f1c1144b640 1 -- 172.21.15.5:0/543779842 >> v2:172.21.15.80:3300/0 conn(0x7f1c0c1453a0 msgr2=0x7f1c0c145810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:41.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.109+0000 7f1c1144b640 1 --2- 172.21.15.5:0/543779842 >> v2:172.21.15.80:3300/0 conn(0x7f1c0c1453a0 0x7f1c0c145810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:41.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.109+0000 7f1c1144b640 1 -- 172.21.15.5:0/543779842 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c0c0b0bc0 con 0x7f1c0c13f260 2024-03-18T15:33:41.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.109+0000 7f1c1144b640 1 --2- 172.21.15.5:0/543779842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c0c13f260 0x7f1c0c13f6a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c0b55f0 tx=0x7f1c0001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f4783b1e311f241 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:41.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.109+0000 7f1c10c4a640 1 -- 172.21.15.5:0/543779842 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c000b2070 con 0x7f1c0c13f260 2024-03-18T15:33:41.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.109+0000 7f1c10c4a640 1 -- 172.21.15.5:0/543779842 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c000a9070 con 0x7f1c0c13f260 2024-03-18T15:33:41.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.109+0000 7f1c10c4a640 1 -- 172.21.15.5:0/543779842 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c000ad880 con 0x7f1c0c13f260 2024-03-18T15:33:41.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.109+0000 7f1c13eb0640 1 -- 172.21.15.5:0/543779842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c0c13f260 msgr2=0x7f1c0c13f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:41.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.109+0000 7f1c13eb0640 1 --2- 172.21.15.5:0/543779842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c0c13f260 0x7f1c0c13f6a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c0b55f0 tx=0x7f1c0001c490 comp rx=0 tx=0).stop 2024-03-18T15:33:41.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.113+0000 7f1c13eb0640 1 -- 172.21.15.5:0/543779842 shutdown_connections 2024-03-18T15:33:41.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.113+0000 7f1c13eb0640 1 --2- 172.21.15.5:0/543779842 >> v2:172.21.15.80:3300/0 conn(0x7f1c0c1453a0 0x7f1c0c145810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:41.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.113+0000 7f1c13eb0640 1 --2- 172.21.15.5:0/543779842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c0c13f260 0x7f1c0c13f6a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:41.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.113+0000 7f1c13eb0640 1 --2- 172.21.15.5:0/543779842 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1c0c06d2d0 0x7f1c0c0b23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:41.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.113+0000 7f1c13eb0640 1 -- 172.21.15.5:0/543779842 >> 172.21.15.5:0/543779842 conn(0x7f1c0c0b5670 msgr2=0x7f1c0c0b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:41.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.113+0000 7f1c13eb0640 1 -- 172.21.15.5:0/543779842 shutdown_connections 2024-03-18T15:33:41.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.113+0000 7f1c13eb0640 1 -- 172.21.15.5:0/543779842 wait complete. 2024-03-18T15:33:41.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.113+0000 7f1c13eb0640 1 Processor -- start 2024-03-18T15:33:41.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.113+0000 7f1c13eb0640 1 -- start start 2024-03-18T15:33:41.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.113+0000 7f1c13eb0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c0c06d2d0 0x7f1c0c13fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:41.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.113+0000 7f1c13eb0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c0c13f260 0x7f1c0c071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:41.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.113+0000 7f1c13eb0640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1c0c1453a0 0x7f1c0c13b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:41.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.113+0000 7f1c13eb0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c0b0960 con 0x7f1c0c06d2d0 2024-03-18T15:33:41.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.113+0000 7f1c13eb0640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c0b07e0 con 0x7f1c0c13f260 2024-03-18T15:33:41.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.113+0000 7f1c13eb0640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f1c0c0b5140 con 0x7f1c0c1453a0 2024-03-18T15:33:41.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.113+0000 7f1c11c4c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c0c06d2d0 0x7f1c0c13fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:41.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.113+0000 7f1c11c4c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c0c06d2d0 0x7f1c0c13fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38196/0 (socket says 172.21.15.5:38196) 2024-03-18T15:33:41.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.113+0000 7f1c11c4c640 1 -- 172.21.15.5:0/1921364527 learned_addr learned my addr 172.21.15.5:0/1921364527 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:41.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.113+0000 7f1c1144b640 1 --2- 172.21.15.5:0/1921364527 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c0c13f260 0x7f1c0c071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:41.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.113+0000 7f1c1244d640 1 --2- 172.21.15.5:0/1921364527 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1c0c1453a0 0x7f1c0c13b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:41.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.113+0000 7f1c11c4c640 1 -- 172.21.15.5:0/1921364527 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1c0c1453a0 msgr2=0x7f1c0c13b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:41.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.113+0000 7f1c11c4c640 1 --2- 172.21.15.5:0/1921364527 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1c0c1453a0 0x7f1c0c13b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:41.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.113+0000 7f1c11c4c640 1 -- 172.21.15.5:0/1921364527 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c0c13f260 msgr2=0x7f1c0c071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:41.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.113+0000 7f1c11c4c640 1 --2- 172.21.15.5:0/1921364527 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c0c13f260 0x7f1c0c071de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:41.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.113+0000 7f1c11c4c640 1 -- 172.21.15.5:0/1921364527 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c0c0715e0 con 0x7f1c0c06d2d0 2024-03-18T15:33:41.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.113+0000 7f1c1144b640 1 --2- 172.21.15.5:0/1921364527 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c0c13f260 0x7f1c0c071de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:33:41.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.113+0000 7f1c1244d640 1 --2- 172.21.15.5:0/1921364527 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1c0c1453a0 0x7f1c0c13b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:33:41.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.113+0000 7f1c11c4c640 1 --2- 172.21.15.5:0/1921364527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c0c06d2d0 0x7f1c0c13fd10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1c0400ef00 tx=0x7f1c0400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:41.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.113+0000 7f1c0a7fc640 1 -- 172.21.15.5:0/1921364527 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c04018020 con 0x7f1c0c06d2d0 2024-03-18T15:33:41.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.113+0000 7f1c0a7fc640 1 -- 172.21.15.5:0/1921364527 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c04014400 con 0x7f1c0c06d2d0 2024-03-18T15:33:41.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.113+0000 7f1c13eb0640 1 -- 172.21.15.5:0/1921364527 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c0c142f40 con 0x7f1c0c06d2d0 2024-03-18T15:33:41.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.113+0000 7f1c0a7fc640 1 -- 172.21.15.5:0/1921364527 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c040147d0 con 0x7f1c0c06d2d0 2024-03-18T15:33:41.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.113+0000 7f1c13eb0640 1 -- 172.21.15.5:0/1921364527 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c0c14fa20 con 0x7f1c0c06d2d0 2024-03-18T15:33:41.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.113+0000 7f1c13eb0640 1 -- 172.21.15.5:0/1921364527 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c0c04f550 con 0x7f1c0c06d2d0 2024-03-18T15:33:41.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.117+0000 7f1c0a7fc640 1 -- 172.21.15.5:0/1921364527 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f1c0401b020 con 0x7f1c0c06d2d0 2024-03-18T15:33:41.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.117+0000 7f1c0a7fc640 1 --2- 172.21.15.5:0/1921364527 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f1bf8067660 0x7f1bf8069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:41.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.117+0000 7f1c0a7fc640 1 -- 172.21.15.5:0/1921364527 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5227+0+0 (secure 0 0 0) 0x7f1c04085e70 con 0x7f1c0c06d2d0 2024-03-18T15:33:41.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.117+0000 7f1c1144b640 1 --2- 172.21.15.5:0/1921364527 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f1bf8067660 0x7f1bf8069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:41.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.117+0000 7f1c1144b640 1 --2- 172.21.15.5:0/1921364527 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f1bf8067660 0x7f1bf8069b20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1c000af000 tx=0x7f1c00008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:41.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.121+0000 7f1c0a7fc640 1 -- 172.21.15.5:0/1921364527 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f1c04022070 con 0x7f1c0c06d2d0 2024-03-18T15:33:41.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.281+0000 7f1c13eb0640 1 -- 172.21.15.5:0/1921364527 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f1c0c143790 con 0x7f1c0c06d2d0 2024-03-18T15:33:41.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.961+0000 7f1c0a7fc640 1 -- 172.21.15.5:0/1921364527 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) ==== 170+0+0 (secure 0 0 0) 0x7f1c04056c30 con 0x7f1c0c06d2d0 2024-03-18T15:33:41.963 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' created 2024-03-18T15:33:41.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.965+0000 7f1c13eb0640 1 -- 172.21.15.5:0/1921364527 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f1bf8067660 msgr2=0x7f1bf8069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:41.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.965+0000 7f1c13eb0640 1 --2- 172.21.15.5:0/1921364527 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f1bf8067660 0x7f1bf8069b20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1c000af000 tx=0x7f1c00008040 comp rx=0 tx=0).stop 2024-03-18T15:33:41.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.965+0000 7f1c13eb0640 1 -- 172.21.15.5:0/1921364527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c0c06d2d0 msgr2=0x7f1c0c13fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:41.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.965+0000 7f1c13eb0640 1 --2- 172.21.15.5:0/1921364527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c0c06d2d0 0x7f1c0c13fd10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1c0400ef00 tx=0x7f1c0400b480 comp rx=0 tx=0).stop 2024-03-18T15:33:41.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.965+0000 7f1c13eb0640 1 -- 172.21.15.5:0/1921364527 shutdown_connections 2024-03-18T15:33:41.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.965+0000 7f1c13eb0640 1 --2- 172.21.15.5:0/1921364527 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f1bf8067660 0x7f1bf8069b20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:41.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.965+0000 7f1c13eb0640 1 --2- 172.21.15.5:0/1921364527 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1c0c1453a0 0x7f1c0c13b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:41.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.965+0000 7f1c13eb0640 1 --2- 172.21.15.5:0/1921364527 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c0c13f260 0x7f1c0c071de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:41.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.965+0000 7f1c13eb0640 1 --2- 172.21.15.5:0/1921364527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c0c06d2d0 0x7f1c0c13fd10 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:41.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.965+0000 7f1c13eb0640 1 -- 172.21.15.5:0/1921364527 >> 172.21.15.5:0/1921364527 conn(0x7f1c0c0b5670 msgr2=0x7f1c0c0b8380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:41.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.965+0000 7f1c13eb0640 1 -- 172.21.15.5:0/1921364527 shutdown_connections 2024-03-18T15:33:41.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:41.965+0000 7f1c13eb0640 1 -- 172.21.15.5:0/1921364527 wait complete. 2024-03-18T15:33:42.002 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-18T15:33:42.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.125+0000 7fa1affff640 1 Processor -- start 2024-03-18T15:33:42.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.125+0000 7fa1affff640 1 -- start start 2024-03-18T15:33:42.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.125+0000 7fa1affff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa1a80b79a0 0x7fa1a806d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:42.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.125+0000 7fa1affff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa1a8141b20 0x7fa1a80b6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:42.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.125+0000 7fa1affff640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1a80b7d80 con 0x7fa1a80b1bc0 2024-03-18T15:33:42.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.125+0000 7fa1affff640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fa1a80b6250 con 0x7fa1a80b79a0 2024-03-18T15:33:42.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.125+0000 7fa1affff640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fa1a80b6530 con 0x7fa1a8141b20 2024-03-18T15:33:42.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.125+0000 7fa1add9b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa1a80b79a0 0x7fa1a806d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:42.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.125+0000 7fa1add9b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa1a80b79a0 0x7fa1a806d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38206/0 (socket says 172.21.15.5:38206) 2024-03-18T15:33:42.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.125+0000 7fa1add9b640 1 -- 172.21.15.5:0/108025916 learned_addr learned my addr 172.21.15.5:0/108025916 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:42.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.125+0000 7fa1ae59c640 1 --2- 172.21.15.5:0/108025916 >> v2:172.21.15.80:3300/0 conn(0x7fa1a8141b20 0x7fa1a80b6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:42.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.125+0000 7fa1add9b640 1 -- 172.21.15.5:0/108025916 >> v2:172.21.15.80:3300/0 conn(0x7fa1a8141b20 msgr2=0x7fa1a80b6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:42.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.125+0000 7fa1add9b640 1 --2- 172.21.15.5:0/108025916 >> v2:172.21.15.80:3300/0 conn(0x7fa1a8141b20 0x7fa1a80b6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:42.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.125+0000 7fa1add9b640 1 -- 172.21.15.5:0/108025916 >> v1:172.21.15.80:6789/0 conn(0x7fa1a80b1bc0 legacy=0x7fa1a8141410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:42.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.125+0000 7fa1add9b640 1 -- 172.21.15.5:0/108025916 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a80b56b0 con 0x7fa1a80b79a0 2024-03-18T15:33:42.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.125+0000 7fa1add9b640 1 --2- 172.21.15.5:0/108025916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1a80b79a0 0x7fa1a806d310 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa1a80b7500 tx=0x7fa1a401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cc2fb4cb6f6816b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:42.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.129+0000 7fa1acd99640 1 -- 172.21.15.5:0/108025916 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a40ab430 con 0x7fa1a80b79a0 2024-03-18T15:33:42.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.129+0000 7fa1acd99640 1 -- 172.21.15.5:0/108025916 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1a40b0070 con 0x7fa1a80b79a0 2024-03-18T15:33:42.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.129+0000 7fa1acd99640 1 -- 172.21.15.5:0/108025916 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a40ab7f0 con 0x7fa1a80b79a0 2024-03-18T15:33:42.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.129+0000 7fa1affff640 1 -- 172.21.15.5:0/108025916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1a80b79a0 msgr2=0x7fa1a806d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:42.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.129+0000 7fa1affff640 1 --2- 172.21.15.5:0/108025916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1a80b79a0 0x7fa1a806d310 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa1a80b7500 tx=0x7fa1a401c520 comp rx=0 tx=0).stop 2024-03-18T15:33:42.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.129+0000 7fa1affff640 1 -- 172.21.15.5:0/108025916 shutdown_connections 2024-03-18T15:33:42.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.129+0000 7fa1affff640 1 --2- 172.21.15.5:0/108025916 >> v2:172.21.15.80:3300/0 conn(0x7fa1a8141b20 0x7fa1a80b6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:42.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.129+0000 7fa1affff640 1 --2- 172.21.15.5:0/108025916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1a80b79a0 0x7fa1a806d310 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:42.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.129+0000 7fa1affff640 1 -- 172.21.15.5:0/108025916 >> 172.21.15.5:0/108025916 conn(0x7fa1a8066fc0 msgr2=0x7fa1a80673d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:42.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.129+0000 7fa1affff640 1 -- 172.21.15.5:0/108025916 shutdown_connections 2024-03-18T15:33:42.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.129+0000 7fa1affff640 1 -- 172.21.15.5:0/108025916 wait complete. 2024-03-18T15:33:42.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.129+0000 7fa1affff640 1 Processor -- start 2024-03-18T15:33:42.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.129+0000 7fa1affff640 1 -- start start 2024-03-18T15:33:42.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.129+0000 7fa1affff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1a80b79a0 0x7fa1a80b9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:42.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.129+0000 7fa1affff640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa1a8141b20 0x7fa1a81e2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:42.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.129+0000 7fa1affff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa1a81e3150 0x7fa1a813fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:42.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.129+0000 7fa1affff640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a81548b0 con 0x7fa1a80b79a0 2024-03-18T15:33:42.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.129+0000 7fa1affff640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a8072cf0 con 0x7fa1a81e3150 2024-03-18T15:33:42.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.129+0000 7fa1affff640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fa1a8154a30 con 0x7fa1a8141b20 2024-03-18T15:33:42.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.129+0000 7fa1add9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1a80b79a0 0x7fa1a80b9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:42.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.129+0000 7fa1add9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1a80b79a0 0x7fa1a80b9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38218/0 (socket says 172.21.15.5:38218) 2024-03-18T15:33:42.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.129+0000 7fa1add9b640 1 -- 172.21.15.5:0/2762881570 learned_addr learned my addr 172.21.15.5:0/2762881570 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:42.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.129+0000 7fa1ae59c640 1 --2- 172.21.15.5:0/2762881570 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa1a81e3150 0x7fa1a813fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:42.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.129+0000 7fa1ad59a640 1 --2- 172.21.15.5:0/2762881570 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa1a8141b20 0x7fa1a81e2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:42.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.129+0000 7fa1add9b640 1 -- 172.21.15.5:0/2762881570 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa1a8141b20 msgr2=0x7fa1a81e2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:42.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.129+0000 7fa1add9b640 1 --2- 172.21.15.5:0/2762881570 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa1a8141b20 0x7fa1a81e2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:42.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.129+0000 7fa1add9b640 1 -- 172.21.15.5:0/2762881570 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa1a81e3150 msgr2=0x7fa1a813fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:42.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.129+0000 7fa1add9b640 1 --2- 172.21.15.5:0/2762881570 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa1a81e3150 0x7fa1a813fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:42.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.129+0000 7fa1add9b640 1 -- 172.21.15.5:0/2762881570 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a8141f50 con 0x7fa1a80b79a0 2024-03-18T15:33:42.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.129+0000 7fa1ad59a640 1 --2- 172.21.15.5:0/2762881570 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa1a8141b20 0x7fa1a81e2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:33:42.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.129+0000 7fa1add9b640 1 --2- 172.21.15.5:0/2762881570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1a80b79a0 0x7fa1a80b9f40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4019130 tx=0x7fa1a401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:42.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.129+0000 7fa19e7fc640 1 -- 172.21.15.5:0/2762881570 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a40b4800 con 0x7fa1a80b79a0 2024-03-18T15:33:42.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.129+0000 7fa19e7fc640 1 -- 172.21.15.5:0/2762881570 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1a40b49a0 con 0x7fa1a80b79a0 2024-03-18T15:33:42.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.129+0000 7fa19e7fc640 1 -- 172.21.15.5:0/2762881570 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a40b4d70 con 0x7fa1a80b79a0 2024-03-18T15:33:42.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.129+0000 7fa1affff640 1 -- 172.21.15.5:0/2762881570 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a813bbe0 con 0x7fa1a80b79a0 2024-03-18T15:33:42.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.129+0000 7fa1affff640 1 -- 172.21.15.5:0/2762881570 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a8142230 con 0x7fa1a80b79a0 2024-03-18T15:33:42.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.133+0000 7fa1affff640 1 -- 172.21.15.5:0/2762881570 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1a804f550 con 0x7fa1a80b79a0 2024-03-18T15:33:42.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.133+0000 7fa19e7fc640 1 -- 172.21.15.5:0/2762881570 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fa1a40b3380 con 0x7fa1a80b79a0 2024-03-18T15:33:42.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.133+0000 7fa19e7fc640 1 --2- 172.21.15.5:0/2762881570 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa18c067660 0x7fa18c069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:42.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.133+0000 7fa19e7fc640 1 -- 172.21.15.5:0/2762881570 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5609+0+0 (secure 0 0 0) 0x7fa1a40a8070 con 0x7fa1a80b79a0 2024-03-18T15:33:42.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.133+0000 7fa1ad59a640 1 --2- 172.21.15.5:0/2762881570 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa18c067660 0x7fa18c069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:42.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.133+0000 7fa1ad59a640 1 --2- 172.21.15.5:0/2762881570 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa18c067660 0x7fa18c069b20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8068f60 tx=0x7fa198009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:42.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.137+0000 7fa19e7fc640 1 -- 172.21.15.5:0/2762881570 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fa1a4016110 con 0x7fa1a80b79a0 2024-03-18T15:33:42.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.333+0000 7fa1affff640 1 -- 172.21.15.5:0/2762881570 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fa1a80723d0 con 0x7fa1a80b79a0 2024-03-18T15:33:42.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:42.989+0000 7fa19e7fc640 1 -- 172.21.15.5:0/2762881570 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7fa1a40f06b0 con 0x7fa1a80b79a0 2024-03-18T15:33:42.992 INFO:teuthology.orchestra.run.smithi005.stderr:new fs with metadata pool 2 and data pool 3 2024-03-18T15:33:43.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.001+0000 7fa1affff640 1 -- 172.21.15.5:0/2762881570 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa18c067660 msgr2=0x7fa18c069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:43.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.001+0000 7fa1affff640 1 --2- 172.21.15.5:0/2762881570 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa18c067660 0x7fa18c069b20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8068f60 tx=0x7fa198009040 comp rx=0 tx=0).stop 2024-03-18T15:33:43.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.001+0000 7fa1affff640 1 -- 172.21.15.5:0/2762881570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1a80b79a0 msgr2=0x7fa1a80b9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:43.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.001+0000 7fa1affff640 1 --2- 172.21.15.5:0/2762881570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1a80b79a0 0x7fa1a80b9f40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4019130 tx=0x7fa1a401ad60 comp rx=0 tx=0).stop 2024-03-18T15:33:43.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.005+0000 7fa1affff640 1 -- 172.21.15.5:0/2762881570 shutdown_connections 2024-03-18T15:33:43.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.005+0000 7fa1affff640 1 --2- 172.21.15.5:0/2762881570 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa18c067660 0x7fa18c069b20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:43.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.005+0000 7fa1affff640 1 --2- 172.21.15.5:0/2762881570 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa1a81e3150 0x7fa1a813fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:43.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.005+0000 7fa1affff640 1 --2- 172.21.15.5:0/2762881570 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa1a8141b20 0x7fa1a81e2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:43.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.005+0000 7fa1affff640 1 --2- 172.21.15.5:0/2762881570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa1a80b79a0 0x7fa1a80b9f40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:43.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.005+0000 7fa1affff640 1 -- 172.21.15.5:0/2762881570 >> 172.21.15.5:0/2762881570 conn(0x7fa1a8066fc0 msgr2=0x7fa1a813e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:43.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.005+0000 7fa1affff640 1 -- 172.21.15.5:0/2762881570 shutdown_connections 2024-03-18T15:33:43.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.005+0000 7fa1affff640 1 -- 172.21.15.5:0/2762881570 wait complete. 2024-03-18T15:33:43.036 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:33:43.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.157+0000 7fbf7eaaf640 1 Processor -- start 2024-03-18T15:33:43.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.157+0000 7fbf7eaaf640 1 -- start start 2024-03-18T15:33:43.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.157+0000 7fbf7eaaf640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbf780702b0 0x7fbf781474c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:43.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.157+0000 7fbf7eaaf640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbf78147a00 0x7fbf78151d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:43.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.157+0000 7fbf7eaaf640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf7806b8c0 con 0x7fbf7806b4e0 2024-03-18T15:33:43.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.157+0000 7fbf7eaaf640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fbf7806c0e0 con 0x7fbf780702b0 2024-03-18T15:33:43.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.157+0000 7fbf7eaaf640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fbf780719b0 con 0x7fbf78147a00 2024-03-18T15:33:43.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.157+0000 7fbf77fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbf780702b0 0x7fbf781474c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:43.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.157+0000 7fbf7c84b640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fbf7806b4e0 0x7fbf78141360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:57154/0 (socket says 172.21.15.5:57154) 2024-03-18T15:33:43.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.157+0000 7fbf77fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbf780702b0 0x7fbf781474c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38226/0 (socket says 172.21.15.5:38226) 2024-03-18T15:33:43.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.157+0000 7fbf7c84b640 1 -- 172.21.15.5:0/2320961129 learned_addr learned my addr 172.21.15.5:0/2320961129 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:33:43.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.157+0000 7fbf7d04c640 1 --2- 172.21.15.5:0/2320961129 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbf78147a00 0x7fbf78151d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:43.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.157+0000 7fbf77fff640 1 -- 172.21.15.5:0/2320961129 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbf78147a00 msgr2=0x7fbf78151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:43.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.157+0000 7fbf77fff640 1 --2- 172.21.15.5:0/2320961129 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbf78147a00 0x7fbf78151d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:43.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.157+0000 7fbf77fff640 1 -- 172.21.15.5:0/2320961129 >> v1:172.21.15.5:6789/0 conn(0x7fbf7806b4e0 legacy=0x7fbf78141360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:43.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.157+0000 7fbf77fff640 1 -- 172.21.15.5:0/2320961129 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf7806e800 con 0x7fbf780702b0 2024-03-18T15:33:43.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.157+0000 7fbf777fe640 1 -- 172.21.15.5:0/2320961129 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 513550089 0 0) 0x7fbf7806b8c0 con 0x7fbf7806b4e0 2024-03-18T15:33:43.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.161+0000 7fbf77fff640 1 --2- 172.21.15.5:0/2320961129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf780702b0 0x7fbf781474c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c0097c0 tx=0x7fbf6c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9d822a3b2376fc3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:43.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.161+0000 7fbf777fe640 1 -- 172.21.15.5:0/2320961129 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c0b0070 con 0x7fbf780702b0 2024-03-18T15:33:43.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.161+0000 7fbf777fe640 1 -- 172.21.15.5:0/2320961129 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf6c0ab600 con 0x7fbf780702b0 2024-03-18T15:33:43.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.161+0000 7fbf777fe640 1 -- 172.21.15.5:0/2320961129 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c0ab9d0 con 0x7fbf780702b0 2024-03-18T15:33:43.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.161+0000 7fbf7eaaf640 1 -- 172.21.15.5:0/2320961129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf780702b0 msgr2=0x7fbf781474c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:43.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.161+0000 7fbf7eaaf640 1 --2- 172.21.15.5:0/2320961129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf780702b0 0x7fbf781474c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c0097c0 tx=0x7fbf6c01c410 comp rx=0 tx=0).stop 2024-03-18T15:33:43.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.161+0000 7fbf7eaaf640 1 -- 172.21.15.5:0/2320961129 shutdown_connections 2024-03-18T15:33:43.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.161+0000 7fbf7eaaf640 1 --2- 172.21.15.5:0/2320961129 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbf78147a00 0x7fbf78151d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:43.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.161+0000 7fbf7eaaf640 1 --2- 172.21.15.5:0/2320961129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf780702b0 0x7fbf781474c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:43.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.161+0000 7fbf7eaaf640 1 -- 172.21.15.5:0/2320961129 >> 172.21.15.5:0/2320961129 conn(0x7fbf780b7a80 msgr2=0x7fbf780b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:43.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.161+0000 7fbf7eaaf640 1 -- 172.21.15.5:0/2320961129 shutdown_connections 2024-03-18T15:33:43.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.161+0000 7fbf7eaaf640 1 -- 172.21.15.5:0/2320961129 wait complete. 2024-03-18T15:33:43.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.161+0000 7fbf7eaaf640 1 Processor -- start 2024-03-18T15:33:43.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.161+0000 7fbf7eaaf640 1 -- start start 2024-03-18T15:33:43.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.161+0000 7fbf7eaaf640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbf7806b4e0 0x7fbf780b5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:43.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.161+0000 7fbf7eaaf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf780702b0 0x7fbf781e3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:43.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.161+0000 7fbf7eaaf640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbf78147a00 0x7fbf78137ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:43.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.161+0000 7fbf7eaaf640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbf78154c40 con 0x7fbf780702b0 2024-03-18T15:33:43.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.161+0000 7fbf7eaaf640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fbf78154ac0 con 0x7fbf78147a00 2024-03-18T15:33:43.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.161+0000 7fbf7eaaf640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fbf78154dc0 con 0x7fbf7806b4e0 2024-03-18T15:33:43.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.161+0000 7fbf77fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf780702b0 0x7fbf781e3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:43.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.161+0000 7fbf77fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf780702b0 0x7fbf781e3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38230/0 (socket says 172.21.15.5:38230) 2024-03-18T15:33:43.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.161+0000 7fbf77fff640 1 -- 172.21.15.5:0/2414071967 learned_addr learned my addr 172.21.15.5:0/2414071967 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:43.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.161+0000 7fbf7d04c640 1 --2- 172.21.15.5:0/2414071967 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbf78147a00 0x7fbf78137ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:43.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.161+0000 7fbf7c84b640 1 --2- 172.21.15.5:0/2414071967 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbf7806b4e0 0x7fbf780b5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:43.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.161+0000 7fbf77fff640 1 -- 172.21.15.5:0/2414071967 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbf7806b4e0 msgr2=0x7fbf780b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:43.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.161+0000 7fbf77fff640 1 --2- 172.21.15.5:0/2414071967 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbf7806b4e0 0x7fbf780b5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:43.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.161+0000 7fbf77fff640 1 -- 172.21.15.5:0/2414071967 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbf78147a00 msgr2=0x7fbf78137ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:43.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.161+0000 7fbf77fff640 1 --2- 172.21.15.5:0/2414071967 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbf78147a00 0x7fbf78137ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:43.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.161+0000 7fbf77fff640 1 -- 172.21.15.5:0/2414071967 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf781433d0 con 0x7fbf780702b0 2024-03-18T15:33:43.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.161+0000 7fbf7c84b640 1 --2- 172.21.15.5:0/2414071967 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbf7806b4e0 0x7fbf780b5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:33:43.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.161+0000 7fbf77fff640 1 --2- 172.21.15.5:0/2414071967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf780702b0 0x7fbf781e3450 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c0ae040 tx=0x7fbf6c01ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:43.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.165+0000 7fbf757fa640 1 -- 172.21.15.5:0/2414071967 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c0bc070 con 0x7fbf780702b0 2024-03-18T15:33:43.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.165+0000 7fbf757fa640 1 -- 172.21.15.5:0/2414071967 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf6c0b1040 con 0x7fbf780702b0 2024-03-18T15:33:43.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.165+0000 7fbf757fa640 1 -- 172.21.15.5:0/2414071967 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c0b0050 con 0x7fbf780702b0 2024-03-18T15:33:43.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.165+0000 7fbf7eaaf640 1 -- 172.21.15.5:0/2414071967 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf7813a480 con 0x7fbf780702b0 2024-03-18T15:33:43.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.165+0000 7fbf7eaaf640 1 -- 172.21.15.5:0/2414071967 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf781425a0 con 0x7fbf780702b0 2024-03-18T15:33:43.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.165+0000 7fbf7eaaf640 1 -- 172.21.15.5:0/2414071967 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf7806c310 con 0x7fbf780702b0 2024-03-18T15:33:43.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.165+0000 7fbf757fa640 1 -- 172.21.15.5:0/2414071967 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fbf6c004250 con 0x7fbf780702b0 2024-03-18T15:33:43.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.165+0000 7fbf757fa640 1 --2- 172.21.15.5:0/2414071967 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fbf64067660 0x7fbf64069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:43.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.165+0000 7fbf757fa640 1 -- 172.21.15.5:0/2414071967 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbf6c11f3d0 con 0x7fbf780702b0 2024-03-18T15:33:43.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.165+0000 7fbf7c84b640 1 --2- 172.21.15.5:0/2414071967 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fbf64067660 0x7fbf64069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:43.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.165+0000 7fbf7c84b640 1 --2- 172.21.15.5:0/2414071967 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fbf64067660 0x7fbf64069b20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbf68007fd0 tx=0x7fbf68009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:43.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.169+0000 7fbf757fa640 1 -- 172.21.15.5:0/2414071967 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fbf6c016090 con 0x7fbf780702b0 2024-03-18T15:33:43.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.333+0000 7fbf7eaaf640 1 -- 172.21.15.5:0/2414071967 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbf7804f550 con 0x7fbf780702b0 2024-03-18T15:33:43.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.337+0000 7fbf757fa640 1 -- 172.21.15.5:0/2414071967 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16462 (secure 0 0 0) 0x7fbf6c0f0050 con 0x7fbf780702b0 2024-03-18T15:33:43.341 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:33:43.341 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":20,"fsid":"ef54fe03-5aa7-4eb2-9b1e-66c9cc345b94","created":"2024-03-18T15:33:18.086614+0000","modified":"2024-03-18T15:33:42.338101+0000","last_up_change":"2024-03-18T15:33:29.229911+0000","last_in_change":"2024-03-18T15:33:20.323840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:28.963186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T15:33:40.533646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T15:33:41.323519+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df1c754b-af59-4d16-8919-fbb6386b3c48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6804","nonce":976264754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6808","nonce":976264754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6815","nonce":976264754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6811","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6813","nonce":976264754}]},"public_addr":"172.21.15.5:6804/976264754","cluster_addr":"172.21.15.5:6808/976264754","heartbeat_back_addr":"172.21.15.5:6815/976264754","heartbeat_front_addr":"172.21.15.5:6813/976264754","state":["exists","up"]},{"osd":1,"uuid":"b9a08b5e-8af4-4144-bdba-d69ed1ac627a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6801","nonce":3901761126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6803","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6805","nonce":3901761126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6812","nonce":3901761126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6807","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6809","nonce":3901761126}]},"public_addr":"172.21.15.5:6801/3901761126","cluster_addr":"172.21.15.5:6805/3901761126","heartbeat_back_addr":"172.21.15.5:6812/3901761126","heartbeat_front_addr":"172.21.15.5:6809/3901761126","state":["exists","up"]},{"osd":2,"uuid":"c3e8bb76-e382-439d-b380-62ef92f1255a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6817","nonce":502968448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6819","nonce":502968448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6823","nonce":502968448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6821","nonce":502968448}]},"public_addr":"172.21.15.5:6817/502968448","cluster_addr":"172.21.15.5:6819/502968448","heartbeat_back_addr":"172.21.15.5:6823/502968448","heartbeat_front_addr":"172.21.15.5:6821/502968448","state":["exists","up"]},{"osd":3,"uuid":"e8c94625-f2cd-4eba-b53d-2e7356e1c02a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6825","nonce":2425932735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6827","nonce":2425932735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6831","nonce":2425932735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6829","nonce":2425932735}]},"public_addr":"172.21.15.5:6825/2425932735","cluster_addr":"172.21.15.5:6827/2425932735","heartbeat_back_addr":"172.21.15.5:6831/2425932735","heartbeat_front_addr":"172.21.15.5:6829/2425932735","state":["exists","up"]},{"osd":4,"uuid":"0099dd52-6265-4d32-82e4-3ef16e73add0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6817","nonce":976514285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6819","nonce":976514285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6823","nonce":976514285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6821","nonce":976514285}]},"public_addr":"172.21.15.80:6817/976514285","cluster_addr":"172.21.15.80:6819/976514285","heartbeat_back_addr":"172.21.15.80:6823/976514285","heartbeat_front_addr":"172.21.15.80:6821/976514285","state":["exists","up"]},{"osd":5,"uuid":"ed10c84f-8c39-41cc-9b64-7c482a94a48e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6825","nonce":627153447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6827","nonce":627153447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6831","nonce":627153447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6829","nonce":627153447}]},"public_addr":"172.21.15.80:6825/627153447","cluster_addr":"172.21.15.80:6827/627153447","heartbeat_back_addr":"172.21.15.80:6831/627153447","heartbeat_front_addr":"172.21.15.80:6829/627153447","state":["exists","up"]},{"osd":6,"uuid":"c7a3538a-7048-4fba-8781-b440f03c25db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6803","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6805","nonce":488342527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6807","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6809","nonce":488342527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6815","nonce":488342527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6811","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6813","nonce":488342527}]},"public_addr":"172.21.15.80:6805/488342527","cluster_addr":"172.21.15.80:6809/488342527","heartbeat_back_addr":"172.21.15.80:6815/488342527","heartbeat_front_addr":"172.21.15.80:6813/488342527","state":["exists","up"]},{"osd":7,"uuid":"f9418104-3a97-4110-bc3a-1f8151d0f757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6801","nonce":982831666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6804","nonce":982831666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6812","nonce":982831666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6808","nonce":982831666}]},"public_addr":"172.21.15.80:6801/982831666","cluster_addr":"172.21.15.80:6804/982831666","heartbeat_back_addr":"172.21.15.80:6812/982831666","heartbeat_front_addr":"172.21.15.80:6808/982831666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.970828+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.858116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.987556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.864603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.756293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.591502+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:33:43.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.337+0000 7fbf7eaaf640 1 -- 172.21.15.5:0/2414071967 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fbf64067660 msgr2=0x7fbf64069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:43.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.337+0000 7fbf7eaaf640 1 --2- 172.21.15.5:0/2414071967 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fbf64067660 0x7fbf64069b20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbf68007fd0 tx=0x7fbf68009040 comp rx=0 tx=0).stop 2024-03-18T15:33:43.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.337+0000 7fbf7eaaf640 1 -- 172.21.15.5:0/2414071967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf780702b0 msgr2=0x7fbf781e3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:43.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.337+0000 7fbf7eaaf640 1 --2- 172.21.15.5:0/2414071967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf780702b0 0x7fbf781e3450 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c0ae040 tx=0x7fbf6c01ac50 comp rx=0 tx=0).stop 2024-03-18T15:33:43.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.337+0000 7fbf7eaaf640 1 -- 172.21.15.5:0/2414071967 shutdown_connections 2024-03-18T15:33:43.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.337+0000 7fbf7eaaf640 1 --2- 172.21.15.5:0/2414071967 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fbf64067660 0x7fbf64069b20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:43.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.337+0000 7fbf7eaaf640 1 --2- 172.21.15.5:0/2414071967 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbf78147a00 0x7fbf78137ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:43.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.337+0000 7fbf7eaaf640 1 --2- 172.21.15.5:0/2414071967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf780702b0 0x7fbf781e3450 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:43.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.337+0000 7fbf7eaaf640 1 --2- 172.21.15.5:0/2414071967 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbf7806b4e0 0x7fbf780b5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:43.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.337+0000 7fbf7eaaf640 1 -- 172.21.15.5:0/2414071967 >> 172.21.15.5:0/2414071967 conn(0x7fbf780b7a80 msgr2=0x7fbf78071ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:43.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.337+0000 7fbf7eaaf640 1 -- 172.21.15.5:0/2414071967 shutdown_connections 2024-03-18T15:33:43.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.341+0000 7fbf7eaaf640 1 -- 172.21.15.5:0/2414071967 wait complete. 2024-03-18T15:33:43.359 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:33:43.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.489+0000 7fe59c426640 1 Processor -- start 2024-03-18T15:33:43.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.489+0000 7fe59c426640 1 -- start start 2024-03-18T15:33:43.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.493+0000 7fe59c426640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe594141700 0x7fe594152de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:43.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.493+0000 7fe59c426640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5940ba420 con 0x7fe59414aa40 2024-03-18T15:33:43.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.493+0000 7fe59c426640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5940b44c0 con 0x7fe5940ba040 2024-03-18T15:33:43.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.493+0000 7fe59c426640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fe5940b4fa0 con 0x7fe594141700 2024-03-18T15:33:43.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.493+0000 7fe59a9c3640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe594141700 0x7fe594152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:43.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.493+0000 7fe5999c1640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe59414aa40 0x7fe594140ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:57170/0 (socket says 172.21.15.5:57170) 2024-03-18T15:33:43.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.493+0000 7fe5999c1640 1 -- 172.21.15.5:0/871737565 learned_addr learned my addr 172.21.15.5:0/871737565 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:33:43.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.493+0000 7fe59a9c3640 1 -- 172.21.15.5:0/871737565 >> v1:172.21.15.80:6789/0 conn(0x7fe5940ba040 legacy=0x7fe5940b5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:43.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.493+0000 7fe59a9c3640 1 -- 172.21.15.5:0/871737565 >> v1:172.21.15.5:6789/0 conn(0x7fe59414aa40 legacy=0x7fe594140ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:43.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.493+0000 7fe5991c0640 1 -- 172.21.15.5:0/871737565 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 872258158 0 0) 0x7fe5940ba420 con 0x7fe59414aa40 2024-03-18T15:33:43.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.493+0000 7fe59a9c3640 1 -- 172.21.15.5:0/871737565 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5940b47a0 con 0x7fe594141700 2024-03-18T15:33:43.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.493+0000 7fe59a9c3640 1 --2- 172.21.15.5:0/871737565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe594141700 0x7fe594152de0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe58c009d90 tx=0x7fe58c01c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7992b97839540b48 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:43.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.493+0000 7fe5991c0640 1 -- 172.21.15.5:0/871737565 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58c0b2070 con 0x7fe594141700 2024-03-18T15:33:43.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.493+0000 7fe5991c0640 1 -- 172.21.15.5:0/871737565 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe58c0a5800 con 0x7fe594141700 2024-03-18T15:33:43.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.493+0000 7fe5991c0640 1 -- 172.21.15.5:0/871737565 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58c0a5bd0 con 0x7fe594141700 2024-03-18T15:33:43.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.497+0000 7fe59c426640 1 -- 172.21.15.5:0/871737565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe594141700 msgr2=0x7fe594152de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:43.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.497+0000 7fe59c426640 1 --2- 172.21.15.5:0/871737565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe594141700 0x7fe594152de0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe58c009d90 tx=0x7fe58c01c850 comp rx=0 tx=0).stop 2024-03-18T15:33:43.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.497+0000 7fe59c426640 1 -- 172.21.15.5:0/871737565 shutdown_connections 2024-03-18T15:33:43.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.497+0000 7fe59c426640 1 --2- 172.21.15.5:0/871737565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe594141700 0x7fe594152de0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:43.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.497+0000 7fe59c426640 1 -- 172.21.15.5:0/871737565 >> 172.21.15.5:0/871737565 conn(0x7fe5940673c0 msgr2=0x7fe5940677d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:43.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.497+0000 7fe59c426640 1 -- 172.21.15.5:0/871737565 shutdown_connections 2024-03-18T15:33:43.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.497+0000 7fe59c426640 1 -- 172.21.15.5:0/871737565 wait complete. 2024-03-18T15:33:43.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.497+0000 7fe59c426640 1 Processor -- start 2024-03-18T15:33:43.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.497+0000 7fe59c426640 1 -- start start 2024-03-18T15:33:43.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.497+0000 7fe59c426640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5940ba040 0x7fe59413f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:43.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.497+0000 7fe59c426640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe594141700 0x7fe59413e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:43.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.497+0000 7fe59c426640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe59414aa40 0x7fe59413c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:43.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.497+0000 7fe59c426640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe594155cc0 con 0x7fe5940ba040 2024-03-18T15:33:43.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.497+0000 7fe59c426640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe594155b40 con 0x7fe59414aa40 2024-03-18T15:33:43.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.497+0000 7fe59c426640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fe594155e40 con 0x7fe594141700 2024-03-18T15:33:43.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.497+0000 7fe59a1c2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5940ba040 0x7fe59413f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:43.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.497+0000 7fe59a1c2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5940ba040 0x7fe59413f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38250/0 (socket says 172.21.15.5:38250) 2024-03-18T15:33:43.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.497+0000 7fe59a1c2640 1 -- 172.21.15.5:0/2395057381 learned_addr learned my addr 172.21.15.5:0/2395057381 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:43.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.497+0000 7fe59a9c3640 1 --2- 172.21.15.5:0/2395057381 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe59414aa40 0x7fe59413c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:43.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.497+0000 7fe5999c1640 1 --2- 172.21.15.5:0/2395057381 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe594141700 0x7fe59413e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:43.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.497+0000 7fe59a1c2640 1 -- 172.21.15.5:0/2395057381 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe594141700 msgr2=0x7fe59413e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:43.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.497+0000 7fe59a1c2640 1 --2- 172.21.15.5:0/2395057381 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe594141700 0x7fe59413e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:43.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.497+0000 7fe59a1c2640 1 -- 172.21.15.5:0/2395057381 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe59414aa40 msgr2=0x7fe59413c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:43.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.497+0000 7fe59a1c2640 1 --2- 172.21.15.5:0/2395057381 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe59414aa40 0x7fe59413c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:43.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.497+0000 7fe59a1c2640 1 -- 172.21.15.5:0/2395057381 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe594144b40 con 0x7fe5940ba040 2024-03-18T15:33:43.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.497+0000 7fe5999c1640 1 --2- 172.21.15.5:0/2395057381 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe594141700 0x7fe59413e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:33:43.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.497+0000 7fe59a1c2640 1 --2- 172.21.15.5:0/2395057381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5940ba040 0x7fe59413f940 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe59000c500 tx=0x7fe59000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:43.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.497+0000 7fe58affd640 1 -- 172.21.15.5:0/2395057381 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe590016020 con 0x7fe5940ba040 2024-03-18T15:33:43.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.497+0000 7fe58affd640 1 -- 172.21.15.5:0/2395057381 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe590005220 con 0x7fe5940ba040 2024-03-18T15:33:43.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.497+0000 7fe59c426640 1 -- 172.21.15.5:0/2395057381 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe59413f1c0 con 0x7fe5940ba040 2024-03-18T15:33:43.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.497+0000 7fe58affd640 1 -- 172.21.15.5:0/2395057381 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe590007030 con 0x7fe5940ba040 2024-03-18T15:33:43.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.497+0000 7fe59c426640 1 -- 172.21.15.5:0/2395057381 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe59413ffe0 con 0x7fe5940ba040 2024-03-18T15:33:43.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.497+0000 7fe59c426640 1 -- 172.21.15.5:0/2395057381 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe59404f550 con 0x7fe5940ba040 2024-03-18T15:33:43.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.501+0000 7fe58affd640 1 -- 172.21.15.5:0/2395057381 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fe5900071f0 con 0x7fe5940ba040 2024-03-18T15:33:43.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.501+0000 7fe58affd640 1 --2- 172.21.15.5:0/2395057381 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe578067660 0x7fe578069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:43.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.501+0000 7fe58affd640 1 -- 172.21.15.5:0/2395057381 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe590084ce0 con 0x7fe5940ba040 2024-03-18T15:33:43.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.501+0000 7fe5999c1640 1 --2- 172.21.15.5:0/2395057381 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe578067660 0x7fe578069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:43.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.501+0000 7fe5999c1640 1 --2- 172.21.15.5:0/2395057381 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe578067660 0x7fe578069b20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe594145220 tx=0x7fe584009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:43.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.505+0000 7fe58affd640 1 -- 172.21.15.5:0/2395057381 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fe590055970 con 0x7fe5940ba040 2024-03-18T15:33:43.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.665+0000 7fe59c426640 1 -- 172.21.15.5:0/2395057381 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe5940bb650 con 0x7fe5940ba040 2024-03-18T15:33:43.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.669+0000 7fe58affd640 1 -- 172.21.15.5:0/2395057381 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16462 (secure 0 0 0) 0x7fe590055270 con 0x7fe5940ba040 2024-03-18T15:33:43.671 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:33:43.672 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":20,"fsid":"ef54fe03-5aa7-4eb2-9b1e-66c9cc345b94","created":"2024-03-18T15:33:18.086614+0000","modified":"2024-03-18T15:33:42.338101+0000","last_up_change":"2024-03-18T15:33:29.229911+0000","last_in_change":"2024-03-18T15:33:20.323840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:28.963186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T15:33:40.533646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T15:33:41.323519+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df1c754b-af59-4d16-8919-fbb6386b3c48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6804","nonce":976264754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6808","nonce":976264754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6815","nonce":976264754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6811","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6813","nonce":976264754}]},"public_addr":"172.21.15.5:6804/976264754","cluster_addr":"172.21.15.5:6808/976264754","heartbeat_back_addr":"172.21.15.5:6815/976264754","heartbeat_front_addr":"172.21.15.5:6813/976264754","state":["exists","up"]},{"osd":1,"uuid":"b9a08b5e-8af4-4144-bdba-d69ed1ac627a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6801","nonce":3901761126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6803","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6805","nonce":3901761126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6812","nonce":3901761126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6807","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6809","nonce":3901761126}]},"public_addr":"172.21.15.5:6801/3901761126","cluster_addr":"172.21.15.5:6805/3901761126","heartbeat_back_addr":"172.21.15.5:6812/3901761126","heartbeat_front_addr":"172.21.15.5:6809/3901761126","state":["exists","up"]},{"osd":2,"uuid":"c3e8bb76-e382-439d-b380-62ef92f1255a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6817","nonce":502968448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6819","nonce":502968448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6823","nonce":502968448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6821","nonce":502968448}]},"public_addr":"172.21.15.5:6817/502968448","cluster_addr":"172.21.15.5:6819/502968448","heartbeat_back_addr":"172.21.15.5:6823/502968448","heartbeat_front_addr":"172.21.15.5:6821/502968448","state":["exists","up"]},{"osd":3,"uuid":"e8c94625-f2cd-4eba-b53d-2e7356e1c02a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6825","nonce":2425932735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6827","nonce":2425932735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6831","nonce":2425932735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6829","nonce":2425932735}]},"public_addr":"172.21.15.5:6825/2425932735","cluster_addr":"172.21.15.5:6827/2425932735","heartbeat_back_addr":"172.21.15.5:6831/2425932735","heartbeat_front_addr":"172.21.15.5:6829/2425932735","state":["exists","up"]},{"osd":4,"uuid":"0099dd52-6265-4d32-82e4-3ef16e73add0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6817","nonce":976514285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6819","nonce":976514285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6823","nonce":976514285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6821","nonce":976514285}]},"public_addr":"172.21.15.80:6817/976514285","cluster_addr":"172.21.15.80:6819/976514285","heartbeat_back_addr":"172.21.15.80:6823/976514285","heartbeat_front_addr":"172.21.15.80:6821/976514285","state":["exists","up"]},{"osd":5,"uuid":"ed10c84f-8c39-41cc-9b64-7c482a94a48e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6825","nonce":627153447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6827","nonce":627153447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6831","nonce":627153447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6829","nonce":627153447}]},"public_addr":"172.21.15.80:6825/627153447","cluster_addr":"172.21.15.80:6827/627153447","heartbeat_back_addr":"172.21.15.80:6831/627153447","heartbeat_front_addr":"172.21.15.80:6829/627153447","state":["exists","up"]},{"osd":6,"uuid":"c7a3538a-7048-4fba-8781-b440f03c25db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6803","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6805","nonce":488342527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6807","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6809","nonce":488342527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6815","nonce":488342527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6811","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6813","nonce":488342527}]},"public_addr":"172.21.15.80:6805/488342527","cluster_addr":"172.21.15.80:6809/488342527","heartbeat_back_addr":"172.21.15.80:6815/488342527","heartbeat_front_addr":"172.21.15.80:6813/488342527","state":["exists","up"]},{"osd":7,"uuid":"f9418104-3a97-4110-bc3a-1f8151d0f757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6801","nonce":982831666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6804","nonce":982831666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6812","nonce":982831666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6808","nonce":982831666}]},"public_addr":"172.21.15.80:6801/982831666","cluster_addr":"172.21.15.80:6804/982831666","heartbeat_back_addr":"172.21.15.80:6812/982831666","heartbeat_front_addr":"172.21.15.80:6808/982831666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.970828+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.858116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.987556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.864603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.756293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.591502+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:33:43.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.673+0000 7fe59c426640 1 -- 172.21.15.5:0/2395057381 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe578067660 msgr2=0x7fe578069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:43.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.673+0000 7fe59c426640 1 --2- 172.21.15.5:0/2395057381 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe578067660 0x7fe578069b20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe594145220 tx=0x7fe584009040 comp rx=0 tx=0).stop 2024-03-18T15:33:43.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.673+0000 7fe59c426640 1 -- 172.21.15.5:0/2395057381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5940ba040 msgr2=0x7fe59413f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:43.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.673+0000 7fe59c426640 1 --2- 172.21.15.5:0/2395057381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5940ba040 0x7fe59413f940 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe59000c500 tx=0x7fe59000c9c0 comp rx=0 tx=0).stop 2024-03-18T15:33:43.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.673+0000 7fe59c426640 1 -- 172.21.15.5:0/2395057381 shutdown_connections 2024-03-18T15:33:43.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.673+0000 7fe59c426640 1 --2- 172.21.15.5:0/2395057381 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe578067660 0x7fe578069b20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:43.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.673+0000 7fe59c426640 1 --2- 172.21.15.5:0/2395057381 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe59414aa40 0x7fe59413c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:43.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.673+0000 7fe59c426640 1 --2- 172.21.15.5:0/2395057381 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe594141700 0x7fe59413e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:43.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.673+0000 7fe59c426640 1 --2- 172.21.15.5:0/2395057381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5940ba040 0x7fe59413f940 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:43.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.673+0000 7fe59c426640 1 -- 172.21.15.5:0/2395057381 >> 172.21.15.5:0/2395057381 conn(0x7fe5940673c0 msgr2=0x7fe5940bbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:43.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.673+0000 7fe59c426640 1 -- 172.21.15.5:0/2395057381 shutdown_connections 2024-03-18T15:33:43.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.673+0000 7fe59c426640 1 -- 172.21.15.5:0/2395057381 wait complete. 2024-03-18T15:33:43.691 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-18T15:33:43.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.829+0000 7f56745b6640 1 Processor -- start 2024-03-18T15:33:43.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.829+0000 7f56745b6640 1 -- start start 2024-03-18T15:33:43.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.829+0000 7f56745b6640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f566c0b2db0 0x7f566c06d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:43.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.829+0000 7f56745b6640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f566c0afb70 con 0x7f566c0af790 2024-03-18T15:33:43.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.829+0000 7f56745b6640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f566c0b4a20 con 0x7f566c072b40 2024-03-18T15:33:43.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.829+0000 7f56745b6640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f566c06d5b0 con 0x7f566c0b2db0 2024-03-18T15:33:43.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.829+0000 7f5672b53640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f566c0b2db0 0x7f566c06d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:43.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.829+0000 7f5672352640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f566c0af790 0x7f566c147780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:57184/0 (socket says 172.21.15.5:57184) 2024-03-18T15:33:43.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.829+0000 7f5672352640 1 -- 172.21.15.5:0/1152938558 learned_addr learned my addr 172.21.15.5:0/1152938558 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:33:43.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.829+0000 7f5672b53640 1 -- 172.21.15.5:0/1152938558 >> v1:172.21.15.80:6789/0 conn(0x7f566c072b40 legacy=0x7f566c0b26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:43.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.829+0000 7f5672b53640 1 -- 172.21.15.5:0/1152938558 >> v1:172.21.15.5:6789/0 conn(0x7f566c0af790 legacy=0x7f566c147780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:43.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.829+0000 7f5672b53640 1 -- 172.21.15.5:0/1152938558 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f566c0b4d00 con 0x7f566c0b2db0 2024-03-18T15:33:43.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.833+0000 7f5672b53640 1 --2- 172.21.15.5:0/1152938558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f566c0b2db0 0x7f566c06d070 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5664009d90 tx=0x7f566401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71a73d2feea8fcb3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:43.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.833+0000 7f5671350640 1 -- 172.21.15.5:0/1152938558 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56640a66b0 con 0x7f566c0b2db0 2024-03-18T15:33:43.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.833+0000 7f5671350640 1 -- 172.21.15.5:0/1152938558 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56640a6850 con 0x7f566c0b2db0 2024-03-18T15:33:43.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.833+0000 7f5671350640 1 -- 172.21.15.5:0/1152938558 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56640a6c20 con 0x7f566c0b2db0 2024-03-18T15:33:43.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.833+0000 7f56745b6640 1 -- 172.21.15.5:0/1152938558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f566c0b2db0 msgr2=0x7f566c06d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:43.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.833+0000 7f56745b6640 1 --2- 172.21.15.5:0/1152938558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f566c0b2db0 0x7f566c06d070 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5664009d90 tx=0x7f566401c8e0 comp rx=0 tx=0).stop 2024-03-18T15:33:43.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.833+0000 7f56745b6640 1 -- 172.21.15.5:0/1152938558 shutdown_connections 2024-03-18T15:33:43.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.833+0000 7f56745b6640 1 --2- 172.21.15.5:0/1152938558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f566c0b2db0 0x7f566c06d070 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:43.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.833+0000 7f56745b6640 1 -- 172.21.15.5:0/1152938558 >> 172.21.15.5:0/1152938558 conn(0x7f566c0b5670 msgr2=0x7f566c0b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:43.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.833+0000 7f56745b6640 1 -- 172.21.15.5:0/1152938558 shutdown_connections 2024-03-18T15:33:43.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.833+0000 7f56745b6640 1 -- 172.21.15.5:0/1152938558 wait complete. 2024-03-18T15:33:43.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.833+0000 7f56745b6640 1 Processor -- start 2024-03-18T15:33:43.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.833+0000 7f56745b6640 1 -- start start 2024-03-18T15:33:43.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.833+0000 7f56745b6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f566c072b40 0x7f566c140bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:43.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.833+0000 7f56745b6640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f566c0af790 0x7f566c071c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:43.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.833+0000 7f56745b6640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f566c0b2db0 0x7f566c0721a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:43.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.833+0000 7f56745b6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f566c0b41d0 con 0x7f566c072b40 2024-03-18T15:33:43.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.833+0000 7f56745b6640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f566c0b4050 con 0x7f566c0af790 2024-03-18T15:33:43.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.833+0000 7f56745b6640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f566c0b4350 con 0x7f566c0b2db0 2024-03-18T15:33:43.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.833+0000 7f5672352640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f566c072b40 0x7f566c140bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:43.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.833+0000 7f5672352640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f566c072b40 0x7f566c140bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38278/0 (socket says 172.21.15.5:38278) 2024-03-18T15:33:43.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.833+0000 7f5672352640 1 -- 172.21.15.5:0/70037572 learned_addr learned my addr 172.21.15.5:0/70037572 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:43.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.833+0000 7f5671b51640 1 --2- 172.21.15.5:0/70037572 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f566c0af790 0x7f566c071c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:43.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.833+0000 7f5672b53640 1 --2- 172.21.15.5:0/70037572 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f566c0b2db0 0x7f566c0721a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:43.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.837+0000 7f5672352640 1 -- 172.21.15.5:0/70037572 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f566c0b2db0 msgr2=0x7f566c0721a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:43.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.837+0000 7f5672352640 1 --2- 172.21.15.5:0/70037572 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f566c0b2db0 0x7f566c0721a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:43.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.837+0000 7f5672352640 1 -- 172.21.15.5:0/70037572 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f566c0af790 msgr2=0x7f566c071c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:43.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.837+0000 7f5672352640 1 --2- 172.21.15.5:0/70037572 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f566c0af790 0x7f566c071c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:43.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.837+0000 7f5672352640 1 -- 172.21.15.5:0/70037572 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f566c141a50 con 0x7f566c072b40 2024-03-18T15:33:43.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.837+0000 7f5671b51640 1 --2- 172.21.15.5:0/70037572 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f566c0af790 0x7f566c071c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:33:43.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.837+0000 7f5672352640 1 --2- 172.21.15.5:0/70037572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f566c072b40 0x7f566c140bd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f566800c530 tx=0x7f566800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:43.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.837+0000 7f5662ffd640 1 -- 172.21.15.5:0/70037572 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5668017070 con 0x7f566c072b40 2024-03-18T15:33:43.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.837+0000 7f5662ffd640 1 -- 172.21.15.5:0/70037572 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5668005cf0 con 0x7f566c072b40 2024-03-18T15:33:43.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.837+0000 7f56745b6640 1 -- 172.21.15.5:0/70037572 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f566c06f720 con 0x7f566c072b40 2024-03-18T15:33:43.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.837+0000 7f5662ffd640 1 -- 172.21.15.5:0/70037572 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5668007270 con 0x7f566c072b40 2024-03-18T15:33:43.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.837+0000 7f56745b6640 1 -- 172.21.15.5:0/70037572 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f566c1de990 con 0x7f566c072b40 2024-03-18T15:33:43.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.837+0000 7f56745b6640 1 -- 172.21.15.5:0/70037572 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f566c04f550 con 0x7f566c072b40 2024-03-18T15:33:43.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.837+0000 7f5662ffd640 1 -- 172.21.15.5:0/70037572 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f566801a070 con 0x7f566c072b40 2024-03-18T15:33:43.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.837+0000 7f5662ffd640 1 --2- 172.21.15.5:0/70037572 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5650067660 0x7f5650069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:43.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.837+0000 7f5662ffd640 1 -- 172.21.15.5:0/70037572 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5668084aa0 con 0x7f566c072b40 2024-03-18T15:33:43.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.837+0000 7f5671b51640 1 --2- 172.21.15.5:0/70037572 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5650067660 0x7f5650069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:43.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.837+0000 7f5671b51640 1 --2- 172.21.15.5:0/70037572 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5650067660 0x7f5650069b20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f566c0af170 tx=0x7f565c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:43.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:43.841+0000 7f5662ffd640 1 -- 172.21.15.5:0/70037572 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f56680550c0 con 0x7f566c072b40 2024-03-18T15:33:44.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:44.041+0000 7f56745b6640 1 -- 172.21.15.5:0/70037572 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f566c0732a0 con 0x7f566c072b40 2024-03-18T15:33:44.549 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-18T15:33:44.552 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-18T15:33:44.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:44.989+0000 7f5662ffd640 1 -- 172.21.15.5:0/70037572 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7f5668059160 con 0x7f566c072b40 2024-03-18T15:33:44.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:44.993+0000 7f56745b6640 1 -- 172.21.15.5:0/70037572 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5650067660 msgr2=0x7f5650069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:44.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:44.993+0000 7f56745b6640 1 --2- 172.21.15.5:0/70037572 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5650067660 0x7f5650069b20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f566c0af170 tx=0x7f565c009040 comp rx=0 tx=0).stop 2024-03-18T15:33:44.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:44.993+0000 7f56745b6640 1 -- 172.21.15.5:0/70037572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f566c072b40 msgr2=0x7f566c140bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:44.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:44.993+0000 7f56745b6640 1 --2- 172.21.15.5:0/70037572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f566c072b40 0x7f566c140bd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f566800c530 tx=0x7f566800c9f0 comp rx=0 tx=0).stop 2024-03-18T15:33:44.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:44.997+0000 7f56745b6640 1 -- 172.21.15.5:0/70037572 shutdown_connections 2024-03-18T15:33:44.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:44.997+0000 7f56745b6640 1 --2- 172.21.15.5:0/70037572 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5650067660 0x7f5650069b20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:44.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:44.997+0000 7f56745b6640 1 --2- 172.21.15.5:0/70037572 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f566c0b2db0 0x7f566c0721a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:44.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:44.997+0000 7f56745b6640 1 --2- 172.21.15.5:0/70037572 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f566c0af790 0x7f566c071c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:44.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:44.997+0000 7f56745b6640 1 --2- 172.21.15.5:0/70037572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f566c072b40 0x7f566c140bd0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:44.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:44.997+0000 7f56745b6640 1 -- 172.21.15.5:0/70037572 >> 172.21.15.5:0/70037572 conn(0x7f566c0b5670 msgr2=0x7f566c0b5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:44.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:44.997+0000 7f56745b6640 1 -- 172.21.15.5:0/70037572 shutdown_connections 2024-03-18T15:33:44.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:44.997+0000 7f56745b6640 1 -- 172.21.15.5:0/70037572 wait complete. 2024-03-18T15:33:45.022 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-18T15:33:45.022 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-18T15:33:45.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.157+0000 7f0ead58d640 1 Processor -- start 2024-03-18T15:33:45.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.157+0000 7f0ead58d640 1 -- start start 2024-03-18T15:33:45.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.161+0000 7f0ead58d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0ea80b79a0 0x7f0ea8148b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:45.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.161+0000 7f0ead58d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0ea806dc40 0x7f0ea81490c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:45.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.161+0000 7f0ead58d640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ea80b7d80 con 0x7f0ea80b56b0 2024-03-18T15:33:45.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.161+0000 7f0ead58d640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f0ea80b6c90 con 0x7f0ea806dc40 2024-03-18T15:33:45.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.161+0000 7f0ead58d640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f0ea80b6f70 con 0x7f0ea80b79a0 2024-03-18T15:33:45.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.161+0000 7f0ea77fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0ea806dc40 0x7f0ea81490c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:45.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.161+0000 7f0ea77fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0ea806dc40 0x7f0ea81490c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38280/0 (socket says 172.21.15.5:38280) 2024-03-18T15:33:45.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.161+0000 7f0ea77fe640 1 -- 172.21.15.5:0/1791590947 learned_addr learned my addr 172.21.15.5:0/1791590947 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:45.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.161+0000 7f0ea6ffd640 1 --2- 172.21.15.5:0/1791590947 >> v2:172.21.15.80:3300/0 conn(0x7f0ea80b79a0 0x7f0ea8148b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:45.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.161+0000 7f0ea77fe640 1 -- 172.21.15.5:0/1791590947 >> v2:172.21.15.80:3300/0 conn(0x7f0ea80b79a0 msgr2=0x7f0ea8148b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:45.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.161+0000 7f0ea77fe640 1 --2- 172.21.15.5:0/1791590947 >> v2:172.21.15.80:3300/0 conn(0x7f0ea80b79a0 0x7f0ea8148b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:45.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.161+0000 7f0ea77fe640 1 -- 172.21.15.5:0/1791590947 >> v1:172.21.15.80:6789/0 conn(0x7f0ea80b56b0 legacy=0x7f0ea806d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:45.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.161+0000 7f0ea77fe640 1 -- 172.21.15.5:0/1791590947 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ea80b6560 con 0x7f0ea806dc40 2024-03-18T15:33:45.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.161+0000 7f0ea77fe640 1 --2- 172.21.15.5:0/1791590947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ea806dc40 0x7f0ea81490c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0ea80725f0 tx=0x7f0ea001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=928af76378c5cf1f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:45.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.161+0000 7f0ea5ffb640 1 -- 172.21.15.5:0/1791590947 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea00ab430 con 0x7f0ea806dc40 2024-03-18T15:33:45.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.161+0000 7f0ea5ffb640 1 -- 172.21.15.5:0/1791590947 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ea00b0070 con 0x7f0ea806dc40 2024-03-18T15:33:45.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.161+0000 7f0ea5ffb640 1 -- 172.21.15.5:0/1791590947 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea00ab7f0 con 0x7f0ea806dc40 2024-03-18T15:33:45.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.165+0000 7f0ead58d640 1 -- 172.21.15.5:0/1791590947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ea806dc40 msgr2=0x7f0ea81490c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:45.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.165+0000 7f0ead58d640 1 --2- 172.21.15.5:0/1791590947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ea806dc40 0x7f0ea81490c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0ea80725f0 tx=0x7f0ea001c520 comp rx=0 tx=0).stop 2024-03-18T15:33:45.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.165+0000 7f0ead58d640 1 -- 172.21.15.5:0/1791590947 shutdown_connections 2024-03-18T15:33:45.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.165+0000 7f0ead58d640 1 --2- 172.21.15.5:0/1791590947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ea806dc40 0x7f0ea81490c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:45.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.165+0000 7f0ead58d640 1 --2- 172.21.15.5:0/1791590947 >> v2:172.21.15.80:3300/0 conn(0x7f0ea80b79a0 0x7f0ea8148b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:45.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.165+0000 7f0ead58d640 1 -- 172.21.15.5:0/1791590947 >> 172.21.15.5:0/1791590947 conn(0x7f0ea8066fc0 msgr2=0x7f0ea80673d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:45.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.165+0000 7f0ead58d640 1 -- 172.21.15.5:0/1791590947 shutdown_connections 2024-03-18T15:33:45.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.165+0000 7f0ead58d640 1 -- 172.21.15.5:0/1791590947 wait complete. 2024-03-18T15:33:45.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.165+0000 7f0ead58d640 1 Processor -- start 2024-03-18T15:33:45.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.165+0000 7f0ead58d640 1 -- start start 2024-03-18T15:33:45.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.165+0000 7f0ead58d640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0ea806dc40 0x7f0ea81e40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:45.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.165+0000 7f0ead58d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ea80b56b0 0x7f0ea814ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:45.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.165+0000 7f0ead58d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ea80b79a0 0x7f0ea813d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:45.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.165+0000 7f0ead58d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea8158cc0 con 0x7f0ea80b79a0 2024-03-18T15:33:45.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.165+0000 7f0ead58d640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea8072cf0 con 0x7f0ea80b56b0 2024-03-18T15:33:45.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.165+0000 7f0ead58d640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f0ea8158e40 con 0x7f0ea806dc40 2024-03-18T15:33:45.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.165+0000 7f0ea77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ea80b79a0 0x7f0ea813d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:45.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.165+0000 7f0ea77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ea80b79a0 0x7f0ea813d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38292/0 (socket says 172.21.15.5:38292) 2024-03-18T15:33:45.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.165+0000 7f0ea77fe640 1 -- 172.21.15.5:0/415723798 learned_addr learned my addr 172.21.15.5:0/415723798 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:45.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.165+0000 7f0ea67fc640 1 --2- 172.21.15.5:0/415723798 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ea80b56b0 0x7f0ea814ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:45.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.165+0000 7f0ea6ffd640 1 --2- 172.21.15.5:0/415723798 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0ea806dc40 0x7f0ea81e40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:45.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.165+0000 7f0ea77fe640 1 -- 172.21.15.5:0/415723798 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0ea806dc40 msgr2=0x7f0ea81e40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:45.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.165+0000 7f0ea77fe640 1 --2- 172.21.15.5:0/415723798 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0ea806dc40 0x7f0ea81e40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:45.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.165+0000 7f0ea77fe640 1 -- 172.21.15.5:0/415723798 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ea80b56b0 msgr2=0x7f0ea814ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:45.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.165+0000 7f0ea77fe640 1 --2- 172.21.15.5:0/415723798 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ea80b56b0 0x7f0ea814ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:45.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.165+0000 7f0ea77fe640 1 -- 172.21.15.5:0/415723798 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ea80b5f40 con 0x7f0ea80b79a0 2024-03-18T15:33:45.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.165+0000 7f0ea77fe640 1 --2- 172.21.15.5:0/415723798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ea80b79a0 0x7f0ea813d1c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0ea00ad040 tx=0x7f0ea001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:45.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.165+0000 7f0e9f7fe640 1 -- 172.21.15.5:0/415723798 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea00b4800 con 0x7f0ea80b79a0 2024-03-18T15:33:45.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.165+0000 7f0e9f7fe640 1 -- 172.21.15.5:0/415723798 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ea00b49c0 con 0x7f0ea80b79a0 2024-03-18T15:33:45.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.165+0000 7f0ead58d640 1 -- 172.21.15.5:0/415723798 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ea80b37f0 con 0x7f0ea80b79a0 2024-03-18T15:33:45.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.165+0000 7f0e9f7fe640 1 -- 172.21.15.5:0/415723798 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea00b4db0 con 0x7f0ea80b79a0 2024-03-18T15:33:45.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.165+0000 7f0ead58d640 1 -- 172.21.15.5:0/415723798 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ea814c410 con 0x7f0ea80b79a0 2024-03-18T15:33:45.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.169+0000 7f0ead58d640 1 -- 172.21.15.5:0/415723798 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ea80b40f0 con 0x7f0ea80b79a0 2024-03-18T15:33:45.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.169+0000 7f0e9f7fe640 1 -- 172.21.15.5:0/415723798 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f0ea00b3380 con 0x7f0ea80b79a0 2024-03-18T15:33:45.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.169+0000 7f0e9f7fe640 1 --2- 172.21.15.5:0/415723798 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0e88067660 0x7f0e88069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:45.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.169+0000 7f0e9f7fe640 1 -- 172.21.15.5:0/415723798 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0ea00a8070 con 0x7f0ea80b79a0 2024-03-18T15:33:45.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.169+0000 7f0ea6ffd640 1 --2- 172.21.15.5:0/415723798 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0e88067660 0x7f0e88069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:45.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.169+0000 7f0ea6ffd640 1 --2- 172.21.15.5:0/415723798 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0e88067660 0x7f0e88069b20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0e98009e90 tx=0x7f0e9800b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:45.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.173+0000 7f0e9f7fe640 1 -- 172.21.15.5:0/415723798 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f0ea0016110 con 0x7f0ea80b79a0 2024-03-18T15:33:45.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.373+0000 7f0ead58d640 1 -- 172.21.15.5:0/415723798 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f0ea804f550 con 0x7f0ea80b79a0 2024-03-18T15:33:46.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:45.997+0000 7f0e9f7fe640 1 -- 172.21.15.5:0/415723798 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7f0ea00f0a60 con 0x7f0ea80b79a0 2024-03-18T15:33:46.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.001+0000 7f0ead58d640 1 -- 172.21.15.5:0/415723798 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0e88067660 msgr2=0x7f0e88069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:46.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.001+0000 7f0ead58d640 1 --2- 172.21.15.5:0/415723798 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0e88067660 0x7f0e88069b20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0e98009e90 tx=0x7f0e9800b040 comp rx=0 tx=0).stop 2024-03-18T15:33:46.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.001+0000 7f0ead58d640 1 -- 172.21.15.5:0/415723798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ea80b79a0 msgr2=0x7f0ea813d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:46.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.001+0000 7f0ead58d640 1 --2- 172.21.15.5:0/415723798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ea80b79a0 0x7f0ea813d1c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0ea00ad040 tx=0x7f0ea001ad60 comp rx=0 tx=0).stop 2024-03-18T15:33:46.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.001+0000 7f0ead58d640 1 -- 172.21.15.5:0/415723798 shutdown_connections 2024-03-18T15:33:46.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.001+0000 7f0ead58d640 1 --2- 172.21.15.5:0/415723798 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0e88067660 0x7f0e88069b20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:46.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.001+0000 7f0ead58d640 1 --2- 172.21.15.5:0/415723798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ea80b79a0 0x7f0ea813d1c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:46.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.001+0000 7f0ead58d640 1 --2- 172.21.15.5:0/415723798 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ea80b56b0 0x7f0ea814ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:46.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.001+0000 7f0ead58d640 1 --2- 172.21.15.5:0/415723798 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0ea806dc40 0x7f0ea81e40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:46.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.001+0000 7f0ead58d640 1 -- 172.21.15.5:0/415723798 >> 172.21.15.5:0/415723798 conn(0x7f0ea8066fc0 msgr2=0x7f0ea8071380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:46.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.001+0000 7f0ead58d640 1 -- 172.21.15.5:0/415723798 shutdown_connections 2024-03-18T15:33:46.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.001+0000 7f0ead58d640 1 -- 172.21.15.5:0/415723798 wait complete. 2024-03-18T15:33:46.025 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-18T15:33:46.026 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:33:46.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.153+0000 7f22ad205640 1 Processor -- start 2024-03-18T15:33:46.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.153+0000 7f22ad205640 1 -- start start 2024-03-18T15:33:46.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.153+0000 7f22ad205640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f22a80af790 0x7f22a8072b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:46.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.153+0000 7f22ad205640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22a80afb70 con 0x7f22a80b26a0 2024-03-18T15:33:46.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.153+0000 7f22ad205640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22a80b4a20 con 0x7f22a806d070 2024-03-18T15:33:46.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.153+0000 7f22ad205640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f22a80b4d00 con 0x7f22a80af790 2024-03-18T15:33:46.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.153+0000 7f22a6d9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f22a80af790 0x7f22a8072b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:46.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.153+0000 7f22a759e640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f22a80b26a0 0x7f22a80b2af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:57200/0 (socket says 172.21.15.5:57200) 2024-03-18T15:33:46.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.153+0000 7f22a759e640 1 -- 172.21.15.5:0/1920091456 learned_addr learned my addr 172.21.15.5:0/1920091456 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:33:46.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.153+0000 7f22a5d9b640 1 -- 172.21.15.5:0/1920091456 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 920585605 0 0) 0x7f22a80afb70 con 0x7f22a80b26a0 2024-03-18T15:33:46.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.153+0000 7f22a5d9b640 1 -- 172.21.15.5:0/1920091456 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2290003530 con 0x7f22a80b26a0 2024-03-18T15:33:46.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.153+0000 7f22a6d9d640 1 -- 172.21.15.5:0/1920091456 >> v1:172.21.15.80:6789/0 conn(0x7f22a806d070 legacy=0x7f22a8147780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:46.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.153+0000 7f22a6d9d640 1 -- 172.21.15.5:0/1920091456 >> v1:172.21.15.5:6789/0 conn(0x7f22a80b26a0 legacy=0x7f22a80b2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:46.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.153+0000 7f22a6d9d640 1 -- 172.21.15.5:0/1920091456 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a806d4b0 con 0x7f22a80af790 2024-03-18T15:33:46.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.157+0000 7f22a6d9d640 1 --2- 172.21.15.5:0/1920091456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22a80af790 0x7f22a8072b40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f22980097c0 tx=0x7f229801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4363199230bc65ec server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:46.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.157+0000 7f22a5d9b640 1 -- 172.21.15.5:0/1920091456 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22980b0070 con 0x7f22a80af790 2024-03-18T15:33:46.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.157+0000 7f22a5d9b640 1 -- 172.21.15.5:0/1920091456 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22980ab600 con 0x7f22a80af790 2024-03-18T15:33:46.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.157+0000 7f22a5d9b640 1 -- 172.21.15.5:0/1920091456 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22980ab9b0 con 0x7f22a80af790 2024-03-18T15:33:46.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.157+0000 7f22ad205640 1 -- 172.21.15.5:0/1920091456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22a80af790 msgr2=0x7f22a8072b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:46.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.157+0000 7f22ad205640 1 --2- 172.21.15.5:0/1920091456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22a80af790 0x7f22a8072b40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f22980097c0 tx=0x7f229801c410 comp rx=0 tx=0).stop 2024-03-18T15:33:46.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.157+0000 7f22ad205640 1 -- 172.21.15.5:0/1920091456 shutdown_connections 2024-03-18T15:33:46.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.157+0000 7f22ad205640 1 --2- 172.21.15.5:0/1920091456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22a80af790 0x7f22a8072b40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:46.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.157+0000 7f22ad205640 1 -- 172.21.15.5:0/1920091456 >> 172.21.15.5:0/1920091456 conn(0x7f22a80b5670 msgr2=0x7f22a80b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:46.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.157+0000 7f22ad205640 1 -- 172.21.15.5:0/1920091456 shutdown_connections 2024-03-18T15:33:46.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.157+0000 7f22ad205640 1 -- 172.21.15.5:0/1920091456 wait complete. 2024-03-18T15:33:46.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.157+0000 7f22ad205640 1 Processor -- start 2024-03-18T15:33:46.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.157+0000 7f22ad205640 1 -- start start 2024-03-18T15:33:46.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.161+0000 7f22ad205640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22a806d070 0x7f22a81524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:46.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.161+0000 7f22ad205640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22a80af790 0x7f22a81dfab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:46.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.161+0000 7f22ad205640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f22a80b26a0 0x7f22a81dfff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:46.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.161+0000 7f22ad205640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f22a80b4190 con 0x7f22a80af790 2024-03-18T15:33:46.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.161+0000 7f22ad205640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f22a80b4010 con 0x7f22a806d070 2024-03-18T15:33:46.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.161+0000 7f22ad205640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f22a80b4310 con 0x7f22a80b26a0 2024-03-18T15:33:46.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.161+0000 7f22a659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22a80af790 0x7f22a81dfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:46.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.161+0000 7f22a6d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22a806d070 0x7f22a81524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:46.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.161+0000 7f22a659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22a80af790 0x7f22a81dfab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38302/0 (socket says 172.21.15.5:38302) 2024-03-18T15:33:46.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.161+0000 7f22a659c640 1 -- 172.21.15.5:0/3744352112 learned_addr learned my addr 172.21.15.5:0/3744352112 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:46.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.161+0000 7f22a759e640 1 --2- 172.21.15.5:0/3744352112 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f22a80b26a0 0x7f22a81dfff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:46.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.161+0000 7f22a659c640 1 -- 172.21.15.5:0/3744352112 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f22a80b26a0 msgr2=0x7f22a81dfff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:46.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.161+0000 7f22a659c640 1 --2- 172.21.15.5:0/3744352112 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f22a80b26a0 0x7f22a81dfff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:46.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.161+0000 7f22a659c640 1 -- 172.21.15.5:0/3744352112 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22a806d070 msgr2=0x7f22a81524f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:46.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.161+0000 7f22a659c640 1 --2- 172.21.15.5:0/3744352112 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22a806d070 0x7f22a81524f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:46.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.161+0000 7f22a659c640 1 -- 172.21.15.5:0/3744352112 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a813ef00 con 0x7f22a80af790 2024-03-18T15:33:46.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.161+0000 7f22a659c640 1 --2- 172.21.15.5:0/3744352112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22a80af790 0x7f22a81dfab0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f229c00c370 tx=0x7f229c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:46.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.161+0000 7f228f7fe640 1 -- 172.21.15.5:0/3744352112 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c0052d0 con 0x7f22a80af790 2024-03-18T15:33:46.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.161+0000 7f228f7fe640 1 -- 172.21.15.5:0/3744352112 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f229c005470 con 0x7f22a80af790 2024-03-18T15:33:46.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.161+0000 7f22ad205640 1 -- 172.21.15.5:0/3744352112 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22a813ff30 con 0x7f22a80af790 2024-03-18T15:33:46.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.161+0000 7f228f7fe640 1 -- 172.21.15.5:0/3744352112 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c007260 con 0x7f22a80af790 2024-03-18T15:33:46.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.161+0000 7f22ad205640 1 -- 172.21.15.5:0/3744352112 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22a8072100 con 0x7f22a80af790 2024-03-18T15:33:46.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.161+0000 7f22ad205640 1 -- 172.21.15.5:0/3744352112 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22a804f550 con 0x7f22a80af790 2024-03-18T15:33:46.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.161+0000 7f228f7fe640 1 -- 172.21.15.5:0/3744352112 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f229c007400 con 0x7f22a80af790 2024-03-18T15:33:46.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.161+0000 7f228f7fe640 1 --2- 172.21.15.5:0/3744352112 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f2294067660 0x7f2294069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:46.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.161+0000 7f228f7fe640 1 -- 172.21.15.5:0/3744352112 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f229c084c70 con 0x7f22a80af790 2024-03-18T15:33:46.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.161+0000 7f22a6d9d640 1 --2- 172.21.15.5:0/3744352112 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f2294067660 0x7f2294069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:46.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.165+0000 7f22a6d9d640 1 --2- 172.21.15.5:0/3744352112 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f2294067660 0x7f2294069b20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f22980ad000 tx=0x7f22980a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:46.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.165+0000 7f228f7fe640 1 -- 172.21.15.5:0/3744352112 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f22a813ff30 con 0x7f22a80af790 2024-03-18T15:33:46.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.361+0000 7f22ad205640 1 -- 172.21.15.5:0/3744352112 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22a806ca50 con 0x7f22a80af790 2024-03-18T15:33:46.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.361+0000 7f228f7fe640 1 -- 172.21.15.5:0/3744352112 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+3521 (secure 0 0 0) 0x7f229c055290 con 0x7f22a80af790 2024-03-18T15:33:46.364 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:33:46.364 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/629589896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":629589896},{"type":"v1","addr":"172.21.15.5:6835","nonce":629589896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-18T15:33:42.338926+0000","modified":"2024-03-18T15:33:45.376646+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6833/1236603350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":1236603350},{"type":"v1","addr":"172.21.15.80:6833","nonce":1236603350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4272,"qdb_cluster":[4272]},"id":1}]} 2024-03-18T15:33:46.364 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 8 2024-03-18T15:33:46.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.365+0000 7f22ad205640 1 -- 172.21.15.5:0/3744352112 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f2294067660 msgr2=0x7f2294069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:46.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.365+0000 7f22ad205640 1 --2- 172.21.15.5:0/3744352112 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f2294067660 0x7f2294069b20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f22980ad000 tx=0x7f22980a6000 comp rx=0 tx=0).stop 2024-03-18T15:33:46.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.365+0000 7f22ad205640 1 -- 172.21.15.5:0/3744352112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22a80af790 msgr2=0x7f22a81dfab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:46.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.365+0000 7f22ad205640 1 --2- 172.21.15.5:0/3744352112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22a80af790 0x7f22a81dfab0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f229c00c370 tx=0x7f229c00c830 comp rx=0 tx=0).stop 2024-03-18T15:33:46.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.365+0000 7f22ad205640 1 -- 172.21.15.5:0/3744352112 shutdown_connections 2024-03-18T15:33:46.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.365+0000 7f22ad205640 1 --2- 172.21.15.5:0/3744352112 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f2294067660 0x7f2294069b20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:46.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.365+0000 7f22ad205640 1 --2- 172.21.15.5:0/3744352112 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f22a80b26a0 0x7f22a81dfff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:46.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.365+0000 7f22ad205640 1 --2- 172.21.15.5:0/3744352112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22a80af790 0x7f22a81dfab0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:46.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.365+0000 7f22ad205640 1 --2- 172.21.15.5:0/3744352112 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22a806d070 0x7f22a81524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:46.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.365+0000 7f22ad205640 1 -- 172.21.15.5:0/3744352112 >> 172.21.15.5:0/3744352112 conn(0x7f22a80b5670 msgr2=0x7f22a80b6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:46.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.365+0000 7f22ad205640 1 -- 172.21.15.5:0/3744352112 shutdown_connections 2024-03-18T15:33:46.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.365+0000 7f22ad205640 1 -- 172.21.15.5:0/3744352112 wait complete. 2024-03-18T15:33:46.383 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:33:46.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.497+0000 7f77007fe640 1 Processor -- start 2024-03-18T15:33:46.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.497+0000 7f77007fe640 1 -- start start 2024-03-18T15:33:46.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.501+0000 7f77007fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f76f80747a0 0x7f76f80ade00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:46.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.501+0000 7f77007fe640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76f814ba90 con 0x7f76f80ae5b0 2024-03-18T15:33:46.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.501+0000 7f77007fe640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76f8074b80 con 0x7f76f8147700 2024-03-18T15:33:46.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.501+0000 7f77007fe640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f76f80aed00 con 0x7f76f80747a0 2024-03-18T15:33:46.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.501+0000 7f76fdd99640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f76f80ae5b0 0x7f76f814ac70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:57206/0 (socket says 172.21.15.5:57206) 2024-03-18T15:33:46.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.501+0000 7f76fdd99640 1 -- 172.21.15.5:0/1306422975 learned_addr learned my addr 172.21.15.5:0/1306422975 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:33:46.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.501+0000 7f76fe59a640 1 --2- 172.21.15.5:0/1306422975 >> v2:172.21.15.80:3300/0 conn(0x7f76f80747a0 0x7f76f80ade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:46.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.501+0000 7f76fd598640 1 -- 172.21.15.5:0/1306422975 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1361235334 0 0) 0x7f76f814ba90 con 0x7f76f80ae5b0 2024-03-18T15:33:46.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.501+0000 7f76fd598640 1 -- 172.21.15.5:0/1306422975 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f76e4003530 con 0x7f76f80ae5b0 2024-03-18T15:33:46.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.501+0000 7f76fd598640 1 -- 172.21.15.5:0/1306422975 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3346417975 0 0) 0x7f76e4003530 con 0x7f76f80ae5b0 2024-03-18T15:33:46.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.501+0000 7f76fd598640 1 -- 172.21.15.5:0/1306422975 >> v2:172.21.15.80:3300/0 conn(0x7f76f80747a0 msgr2=0x7f76f80ade00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:46.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.501+0000 7f76fd598640 1 --2- 172.21.15.5:0/1306422975 >> v2:172.21.15.80:3300/0 conn(0x7f76f80747a0 0x7f76f80ade00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:46.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.501+0000 7f76fd598640 1 -- 172.21.15.5:0/1306422975 >> v1:172.21.15.80:6789/0 conn(0x7f76f8147700 legacy=0x7f76f814b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:46.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.501+0000 7f76fd598640 1 -- 172.21.15.5:0/1306422975 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76f80aefe0 con 0x7f76f80ae5b0 2024-03-18T15:33:46.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.501+0000 7f76fd598640 1 -- 172.21.15.5:0/1306422975 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3837549003 0 0) 0x7f76e8001680 con 0x7f76f80ae5b0 2024-03-18T15:33:46.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.501+0000 7f76fd598640 1 -- 172.21.15.5:0/1306422975 >> v1:172.21.15.5:6789/0 conn(0x7f76f80ae5b0 legacy=0x7f76f814ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:46.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.501+0000 7f76fd598640 1 --2- 172.21.15.5:0/1306422975 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f76e4003aa0 0x7f76e4003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:46.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.501+0000 7f76fd598640 1 --2- 172.21.15.5:0/1306422975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76e4004810 0x7f76e4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:46.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.501+0000 7f76fd598640 1 --2- 172.21.15.5:0/1306422975 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f76e40052a0 0x7f76e400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:46.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.501+0000 7f76fd598640 1 -- 172.21.15.5:0/1306422975 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f76f80aed00 con 0x7f76e4004810 2024-03-18T15:33:46.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.501+0000 7f76fd598640 1 -- 172.21.15.5:0/1306422975 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f76e400c1f0 con 0x7f76e40052a0 2024-03-18T15:33:46.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.501+0000 7f76fd598640 1 -- 172.21.15.5:0/1306422975 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f76e400c3f0 con 0x7f76e4003aa0 2024-03-18T15:33:46.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.501+0000 7f76fdd99640 1 --2- 172.21.15.5:0/1306422975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76e4004810 0x7f76e4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:46.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.501+0000 7f76fe59a640 1 --2- 172.21.15.5:0/1306422975 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f76e4003aa0 0x7f76e4003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:46.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.501+0000 7f76fed9b640 1 --2- 172.21.15.5:0/1306422975 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f76e40052a0 0x7f76e400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:46.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.501+0000 7f76fdd99640 1 -- 172.21.15.5:0/1306422975 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f76e4003aa0 msgr2=0x7f76e4003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:46.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.501+0000 7f76fdd99640 1 --2- 172.21.15.5:0/1306422975 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f76e4003aa0 0x7f76e4003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:46.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.501+0000 7f76fdd99640 1 -- 172.21.15.5:0/1306422975 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f76e40052a0 msgr2=0x7f76e400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:46.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.501+0000 7f76fdd99640 1 --2- 172.21.15.5:0/1306422975 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f76e40052a0 0x7f76e400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:46.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.501+0000 7f76fdd99640 1 -- 172.21.15.5:0/1306422975 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76e4003530 con 0x7f76e4004810 2024-03-18T15:33:46.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.501+0000 7f76fed9b640 1 -- 172.21.15.5:0/1306422975 reap_dead start 2024-03-18T15:33:46.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.505+0000 7f76fdd99640 1 --2- 172.21.15.5:0/1306422975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76e4004810 0x7f76e4004c90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f76e800c210 tx=0x7f76e801cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e40b3ee3ac71d11 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:46.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.505+0000 7f76fd598640 1 -- 172.21.15.5:0/1306422975 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e8007d20 con 0x7f76e4004810 2024-03-18T15:33:46.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.505+0000 7f76fd598640 1 -- 172.21.15.5:0/1306422975 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76e80a9070 con 0x7f76e4004810 2024-03-18T15:33:46.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.505+0000 7f76fd598640 1 -- 172.21.15.5:0/1306422975 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e8005da0 con 0x7f76e4004810 2024-03-18T15:33:46.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.505+0000 7f77007fe640 1 -- 172.21.15.5:0/1306422975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76e4004810 msgr2=0x7f76e4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:46.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.505+0000 7f77007fe640 1 --2- 172.21.15.5:0/1306422975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76e4004810 0x7f76e4004c90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f76e800c210 tx=0x7f76e801cc60 comp rx=0 tx=0).stop 2024-03-18T15:33:46.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.505+0000 7f77007fe640 1 -- 172.21.15.5:0/1306422975 shutdown_connections 2024-03-18T15:33:46.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.505+0000 7f77007fe640 1 --2- 172.21.15.5:0/1306422975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76e4004810 0x7f76e4004c90 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:46.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.505+0000 7f77007fe640 1 -- 172.21.15.5:0/1306422975 >> 172.21.15.5:0/1306422975 conn(0x7f76f80b5d70 msgr2=0x7f76f80b6180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:46.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.505+0000 7f77007fe640 1 -- 172.21.15.5:0/1306422975 shutdown_connections 2024-03-18T15:33:46.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.505+0000 7f77007fe640 1 -- 172.21.15.5:0/1306422975 wait complete. 2024-03-18T15:33:46.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.505+0000 7f77007fe640 1 Processor -- start 2024-03-18T15:33:46.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.505+0000 7f77007fe640 1 -- start start 2024-03-18T15:33:46.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.505+0000 7f77007fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76e4004810 0x7f76f80b2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:46.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.505+0000 7f77007fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f76f8143540 0x7f76f81e0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:46.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.505+0000 7f77007fe640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f76f806fc90 0x7f76f813e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:46.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.505+0000 7f77007fe640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f76f80b8d00 con 0x7f76e4004810 2024-03-18T15:33:46.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.505+0000 7f77007fe640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f76f80b8b80 con 0x7f76f8143540 2024-03-18T15:33:46.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.505+0000 7f77007fe640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f76f814b480 con 0x7f76f806fc90 2024-03-18T15:33:46.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.505+0000 7f76fe59a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76e4004810 0x7f76f80b2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:46.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.505+0000 7f76fe59a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76e4004810 0x7f76f80b2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38318/0 (socket says 172.21.15.5:38318) 2024-03-18T15:33:46.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.505+0000 7f76fe59a640 1 -- 172.21.15.5:0/793268282 learned_addr learned my addr 172.21.15.5:0/793268282 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:46.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.505+0000 7f76fdd99640 1 --2- 172.21.15.5:0/793268282 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f76f8143540 0x7f76f81e0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:46.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.505+0000 7f76fed9b640 1 --2- 172.21.15.5:0/793268282 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f76f806fc90 0x7f76f813e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:46.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.505+0000 7f76fe59a640 1 -- 172.21.15.5:0/793268282 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f76f806fc90 msgr2=0x7f76f813e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:46.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.505+0000 7f76fe59a640 1 --2- 172.21.15.5:0/793268282 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f76f806fc90 0x7f76f813e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:46.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.505+0000 7f76fe59a640 1 -- 172.21.15.5:0/793268282 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f76f8143540 msgr2=0x7f76f81e0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:46.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.505+0000 7f76fe59a640 1 --2- 172.21.15.5:0/793268282 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f76f8143540 0x7f76f81e0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:46.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.505+0000 7f76fe59a640 1 -- 172.21.15.5:0/793268282 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76f8071040 con 0x7f76e4004810 2024-03-18T15:33:46.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.505+0000 7f76fdd99640 1 --2- 172.21.15.5:0/793268282 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f76f8143540 0x7f76f81e0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:33:46.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.505+0000 7f76fe59a640 1 --2- 172.21.15.5:0/793268282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76e4004810 0x7f76f80b2c60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f76f4011ad0 tx=0x7f76f4011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:46.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.509+0000 7f76eeffd640 1 -- 172.21.15.5:0/793268282 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f400fbb0 con 0x7f76e4004810 2024-03-18T15:33:46.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.509+0000 7f76eeffd640 1 -- 172.21.15.5:0/793268282 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76f400fd50 con 0x7f76e4004810 2024-03-18T15:33:46.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.509+0000 7f77007fe640 1 -- 172.21.15.5:0/793268282 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76f81e1500 con 0x7f76e4004810 2024-03-18T15:33:46.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.509+0000 7f76eeffd640 1 -- 172.21.15.5:0/793268282 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f4014680 con 0x7f76e4004810 2024-03-18T15:33:46.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.509+0000 7f77007fe640 1 -- 172.21.15.5:0/793268282 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76f81e07c0 con 0x7f76e4004810 2024-03-18T15:33:46.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.509+0000 7f77007fe640 1 -- 172.21.15.5:0/793268282 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76f8070180 con 0x7f76e4004810 2024-03-18T15:33:46.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.509+0000 7f76eeffd640 1 -- 172.21.15.5:0/793268282 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f76f4012040 con 0x7f76e4004810 2024-03-18T15:33:46.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.509+0000 7f76eeffd640 1 --2- 172.21.15.5:0/793268282 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f76dc067690 0x7f76dc069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:46.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.509+0000 7f76eeffd640 1 -- 172.21.15.5:0/793268282 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f76f4085f50 con 0x7f76e4004810 2024-03-18T15:33:46.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.509+0000 7f76fdd99640 1 --2- 172.21.15.5:0/793268282 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f76dc067690 0x7f76dc069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:46.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.509+0000 7f76fdd99640 1 --2- 172.21.15.5:0/793268282 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f76dc067690 0x7f76dc069b50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f76e80ab000 tx=0x7f76e80a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:46.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.513+0000 7f76eeffd640 1 -- 172.21.15.5:0/793268282 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f76f8070180 con 0x7f76e4004810 2024-03-18T15:33:46.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.677+0000 7f77007fe640 1 -- 172.21.15.5:0/793268282 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f76f81e05b0 con 0x7f76e4004810 2024-03-18T15:33:46.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.681+0000 7f76eeffd640 1 -- 172.21.15.5:0/793268282 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16449 (secure 0 0 0) 0x7f76f4019030 con 0x7f76e4004810 2024-03-18T15:33:46.683 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:33:46.683 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":21,"fsid":"ef54fe03-5aa7-4eb2-9b1e-66c9cc345b94","created":"2024-03-18T15:33:18.086614+0000","modified":"2024-03-18T15:33:43.984050+0000","last_up_change":"2024-03-18T15:33:29.229911+0000","last_in_change":"2024-03-18T15:33:20.323840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:28.963186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T15:33:40.533646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T15:33:41.323519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df1c754b-af59-4d16-8919-fbb6386b3c48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6804","nonce":976264754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6808","nonce":976264754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6815","nonce":976264754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6811","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6813","nonce":976264754}]},"public_addr":"172.21.15.5:6804/976264754","cluster_addr":"172.21.15.5:6808/976264754","heartbeat_back_addr":"172.21.15.5:6815/976264754","heartbeat_front_addr":"172.21.15.5:6813/976264754","state":["exists","up"]},{"osd":1,"uuid":"b9a08b5e-8af4-4144-bdba-d69ed1ac627a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6801","nonce":3901761126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6803","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6805","nonce":3901761126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6812","nonce":3901761126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6807","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6809","nonce":3901761126}]},"public_addr":"172.21.15.5:6801/3901761126","cluster_addr":"172.21.15.5:6805/3901761126","heartbeat_back_addr":"172.21.15.5:6812/3901761126","heartbeat_front_addr":"172.21.15.5:6809/3901761126","state":["exists","up"]},{"osd":2,"uuid":"c3e8bb76-e382-439d-b380-62ef92f1255a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6817","nonce":502968448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6819","nonce":502968448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6823","nonce":502968448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6821","nonce":502968448}]},"public_addr":"172.21.15.5:6817/502968448","cluster_addr":"172.21.15.5:6819/502968448","heartbeat_back_addr":"172.21.15.5:6823/502968448","heartbeat_front_addr":"172.21.15.5:6821/502968448","state":["exists","up"]},{"osd":3,"uuid":"e8c94625-f2cd-4eba-b53d-2e7356e1c02a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6825","nonce":2425932735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6827","nonce":2425932735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6831","nonce":2425932735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6829","nonce":2425932735}]},"public_addr":"172.21.15.5:6825/2425932735","cluster_addr":"172.21.15.5:6827/2425932735","heartbeat_back_addr":"172.21.15.5:6831/2425932735","heartbeat_front_addr":"172.21.15.5:6829/2425932735","state":["exists","up"]},{"osd":4,"uuid":"0099dd52-6265-4d32-82e4-3ef16e73add0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6817","nonce":976514285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6819","nonce":976514285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6823","nonce":976514285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6821","nonce":976514285}]},"public_addr":"172.21.15.80:6817/976514285","cluster_addr":"172.21.15.80:6819/976514285","heartbeat_back_addr":"172.21.15.80:6823/976514285","heartbeat_front_addr":"172.21.15.80:6821/976514285","state":["exists","up"]},{"osd":5,"uuid":"ed10c84f-8c39-41cc-9b64-7c482a94a48e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6825","nonce":627153447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6827","nonce":627153447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6831","nonce":627153447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6829","nonce":627153447}]},"public_addr":"172.21.15.80:6825/627153447","cluster_addr":"172.21.15.80:6827/627153447","heartbeat_back_addr":"172.21.15.80:6831/627153447","heartbeat_front_addr":"172.21.15.80:6829/627153447","state":["exists","up"]},{"osd":6,"uuid":"c7a3538a-7048-4fba-8781-b440f03c25db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6803","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6805","nonce":488342527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6807","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6809","nonce":488342527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6815","nonce":488342527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6811","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6813","nonce":488342527}]},"public_addr":"172.21.15.80:6805/488342527","cluster_addr":"172.21.15.80:6809/488342527","heartbeat_back_addr":"172.21.15.80:6815/488342527","heartbeat_front_addr":"172.21.15.80:6813/488342527","state":["exists","up"]},{"osd":7,"uuid":"f9418104-3a97-4110-bc3a-1f8151d0f757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6801","nonce":982831666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6804","nonce":982831666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6812","nonce":982831666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6808","nonce":982831666}]},"public_addr":"172.21.15.80:6801/982831666","cluster_addr":"172.21.15.80:6804/982831666","heartbeat_back_addr":"172.21.15.80:6812/982831666","heartbeat_front_addr":"172.21.15.80:6808/982831666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.970828+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.858116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.987556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.864603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.756293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.591502+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:33:46.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.681+0000 7f77007fe640 1 -- 172.21.15.5:0/793268282 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f76dc067690 msgr2=0x7f76dc069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:46.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.681+0000 7f77007fe640 1 --2- 172.21.15.5:0/793268282 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f76dc067690 0x7f76dc069b50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f76e80ab000 tx=0x7f76e80a7070 comp rx=0 tx=0).stop 2024-03-18T15:33:46.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.681+0000 7f77007fe640 1 -- 172.21.15.5:0/793268282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76e4004810 msgr2=0x7f76f80b2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:46.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.681+0000 7f77007fe640 1 --2- 172.21.15.5:0/793268282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76e4004810 0x7f76f80b2c60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f76f4011ad0 tx=0x7f76f4011f90 comp rx=0 tx=0).stop 2024-03-18T15:33:46.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.681+0000 7f77007fe640 1 -- 172.21.15.5:0/793268282 shutdown_connections 2024-03-18T15:33:46.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.681+0000 7f77007fe640 1 --2- 172.21.15.5:0/793268282 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f76dc067690 0x7f76dc069b50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:46.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.681+0000 7f77007fe640 1 --2- 172.21.15.5:0/793268282 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f76f806fc90 0x7f76f813e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:46.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.681+0000 7f77007fe640 1 --2- 172.21.15.5:0/793268282 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f76f8143540 0x7f76f81e0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:46.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.681+0000 7f77007fe640 1 --2- 172.21.15.5:0/793268282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76e4004810 0x7f76f80b2c60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:46.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.681+0000 7f77007fe640 1 -- 172.21.15.5:0/793268282 >> 172.21.15.5:0/793268282 conn(0x7f76f80b5d70 msgr2=0x7f76f80b6150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:46.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.681+0000 7f77007fe640 1 -- 172.21.15.5:0/793268282 shutdown_connections 2024-03-18T15:33:46.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.681+0000 7f77007fe640 1 -- 172.21.15.5:0/793268282 wait complete. 2024-03-18T15:33:46.700 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-18T15:33:46.701 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-18T15:33:46.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.821+0000 7f6014bf5640 1 Processor -- start 2024-03-18T15:33:46.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.821+0000 7f6014bf5640 1 -- start start 2024-03-18T15:33:46.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.821+0000 7f6014bf5640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f601006b4e0 0x7f60101413f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:46.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.821+0000 7f6014bf5640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6010070320 0x7f6010147530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:46.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.821+0000 7f6014bf5640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f601006b8c0 con 0x7f6010141930 2024-03-18T15:33:46.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.821+0000 7f6014bf5640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f601006ee50 con 0x7f601006b4e0 2024-03-18T15:33:46.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.821+0000 7f6014bf5640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f60100707b0 con 0x7f6010070320 2024-03-18T15:33:46.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.821+0000 7f600e59c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f601006b4e0 0x7f60101413f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:46.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.821+0000 7f600ed9d640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f6010141930 0x7f601006d0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.5:44032/0 (socket says 172.21.15.5:44032) 2024-03-18T15:33:46.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.821+0000 7f600ed9d640 1 -- 172.21.15.5:0/1186404827 learned_addr learned my addr 172.21.15.5:0/1186404827 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:33:46.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.821+0000 7f600dd9b640 1 --2- 172.21.15.5:0/1186404827 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6010070320 0x7f6010147530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:46.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.821+0000 7f600e59c640 1 -- 172.21.15.5:0/1186404827 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6010070320 msgr2=0x7f6010147530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:46.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.821+0000 7f600e59c640 1 --2- 172.21.15.5:0/1186404827 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6010070320 0x7f6010147530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:46.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.821+0000 7f600e59c640 1 -- 172.21.15.5:0/1186404827 >> v1:172.21.15.80:6789/0 conn(0x7f6010141930 legacy=0x7f601006d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:46.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.821+0000 7f600e59c640 1 -- 172.21.15.5:0/1186404827 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f601006eae0 con 0x7f601006b4e0 2024-03-18T15:33:46.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.825+0000 7f600e59c640 1 --2- 172.21.15.5:0/1186404827 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f601006b4e0 0x7f60101413f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f60000097c0 tx=0x7f600001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9de1fd4b47f347d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:46.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.825+0000 7f600d59a640 1 -- 172.21.15.5:0/1186404827 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60000b0070 con 0x7f601006b4e0 2024-03-18T15:33:46.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.825+0000 7f600d59a640 1 -- 172.21.15.5:0/1186404827 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60000ab600 con 0x7f601006b4e0 2024-03-18T15:33:46.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.825+0000 7f600d59a640 1 -- 172.21.15.5:0/1186404827 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60000ab9d0 con 0x7f601006b4e0 2024-03-18T15:33:46.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.825+0000 7f6014bf5640 1 -- 172.21.15.5:0/1186404827 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f601006b4e0 msgr2=0x7f60101413f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:46.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.825+0000 7f6014bf5640 1 --2- 172.21.15.5:0/1186404827 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f601006b4e0 0x7f60101413f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f60000097c0 tx=0x7f600001c410 comp rx=0 tx=0).stop 2024-03-18T15:33:46.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.825+0000 7f6014bf5640 1 -- 172.21.15.5:0/1186404827 shutdown_connections 2024-03-18T15:33:46.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.825+0000 7f6014bf5640 1 --2- 172.21.15.5:0/1186404827 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6010070320 0x7f6010147530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:46.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.825+0000 7f6014bf5640 1 --2- 172.21.15.5:0/1186404827 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f601006b4e0 0x7f60101413f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:46.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.825+0000 7f6014bf5640 1 -- 172.21.15.5:0/1186404827 >> 172.21.15.5:0/1186404827 conn(0x7f60100b7a80 msgr2=0x7f60100b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:46.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.825+0000 7f6014bf5640 1 -- 172.21.15.5:0/1186404827 shutdown_connections 2024-03-18T15:33:46.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.825+0000 7f6014bf5640 1 -- 172.21.15.5:0/1186404827 wait complete. 2024-03-18T15:33:46.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.825+0000 7f6014bf5640 1 Processor -- start 2024-03-18T15:33:46.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.825+0000 7f6014bf5640 1 -- start start 2024-03-18T15:33:46.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.825+0000 7f6014bf5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f601006b4e0 0x7f601013f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:46.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.825+0000 7f6014bf5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6010141930 0x7f60101e01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:46.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.825+0000 7f6014bf5640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f60101e4370 0x7f6010146980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:46.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.825+0000 7f6014bf5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f601006e850 con 0x7f6010141930 2024-03-18T15:33:46.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.825+0000 7f6014bf5640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f601006e6d0 con 0x7f601006b4e0 2024-03-18T15:33:46.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.825+0000 7f6014bf5640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f6010154ce0 con 0x7f60101e4370 2024-03-18T15:33:46.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.825+0000 7f600dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6010141930 0x7f60101e01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:46.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.825+0000 7f600dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6010141930 0x7f60101e01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38334/0 (socket says 172.21.15.5:38334) 2024-03-18T15:33:46.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.825+0000 7f600dd9b640 1 -- 172.21.15.5:0/2678812154 learned_addr learned my addr 172.21.15.5:0/2678812154 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:46.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.825+0000 7f600ed9d640 1 --2- 172.21.15.5:0/2678812154 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f60101e4370 0x7f6010146980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:46.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.825+0000 7f600e59c640 1 --2- 172.21.15.5:0/2678812154 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f601006b4e0 0x7f601013f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:46.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.829+0000 7f600dd9b640 1 -- 172.21.15.5:0/2678812154 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f60101e4370 msgr2=0x7f6010146980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:46.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.829+0000 7f600dd9b640 1 --2- 172.21.15.5:0/2678812154 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f60101e4370 0x7f6010146980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:46.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.829+0000 7f600dd9b640 1 -- 172.21.15.5:0/2678812154 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f601006b4e0 msgr2=0x7f601013f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:46.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.829+0000 7f600dd9b640 1 --2- 172.21.15.5:0/2678812154 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f601006b4e0 0x7f601013f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:46.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.829+0000 7f600dd9b640 1 -- 172.21.15.5:0/2678812154 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60100b61c0 con 0x7f6010141930 2024-03-18T15:33:46.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.829+0000 7f600dd9b640 1 --2- 172.21.15.5:0/2678812154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6010141930 0x7f60101e01a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f600400e930 tx=0x7f600400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:46.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.829+0000 7f5ffeffd640 1 -- 172.21.15.5:0/2678812154 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6004017070 con 0x7f6010141930 2024-03-18T15:33:46.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.829+0000 7f5ffeffd640 1 -- 172.21.15.5:0/2678812154 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f600400ca00 con 0x7f6010141930 2024-03-18T15:33:46.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.829+0000 7f6014bf5640 1 -- 172.21.15.5:0/2678812154 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f601013f820 con 0x7f6010141930 2024-03-18T15:33:46.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.829+0000 7f5ffeffd640 1 -- 172.21.15.5:0/2678812154 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600400cdb0 con 0x7f6010141930 2024-03-18T15:33:46.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.829+0000 7f6014bf5640 1 -- 172.21.15.5:0/2678812154 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60101e3e50 con 0x7f6010141930 2024-03-18T15:33:46.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.829+0000 7f6014bf5640 1 -- 172.21.15.5:0/2678812154 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f601004f550 con 0x7f6010141930 2024-03-18T15:33:46.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.829+0000 7f5ffeffd640 1 -- 172.21.15.5:0/2678812154 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f600401a070 con 0x7f6010141930 2024-03-18T15:33:46.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.829+0000 7f5ffeffd640 1 --2- 172.21.15.5:0/2678812154 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5ff0067660 0x7f5ff0069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:46.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.829+0000 7f5ffeffd640 1 -- 172.21.15.5:0/2678812154 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f6004085900 con 0x7f6010141930 2024-03-18T15:33:46.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.829+0000 7f600e59c640 1 --2- 172.21.15.5:0/2678812154 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5ff0067660 0x7f5ff0069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:46.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.829+0000 7f600e59c640 1 --2- 172.21.15.5:0/2678812154 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5ff0067660 0x7f5ff0069b20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f60000ad000 tx=0x7f6000008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:46.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.833+0000 7f5ffeffd640 1 -- 172.21.15.5:0/2678812154 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f6004055f70 con 0x7f6010141930 2024-03-18T15:33:46.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.985+0000 7f6014bf5640 1 -- 172.21.15.5:0/2678812154 --> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6010071360 con 0x7f5ff0067660 2024-03-18T15:33:46.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.993+0000 7f5ffeffd640 1 -- 172.21.15.5:0/2678812154 <== mgr.4110 v2:172.21.15.5:6832/3362034860 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251883 (secure 0 0 0) 0x7f6010071360 con 0x7f5ff0067660 2024-03-18T15:33:46.995 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:33:46.999 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-03-18T15:33:47.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.997+0000 7f6014bf5640 1 -- 172.21.15.5:0/2678812154 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5ff0067660 msgr2=0x7f5ff0069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:47.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.997+0000 7f6014bf5640 1 --2- 172.21.15.5:0/2678812154 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5ff0067660 0x7f5ff0069b20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f60000ad000 tx=0x7f6000008040 comp rx=0 tx=0).stop 2024-03-18T15:33:47.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.997+0000 7f6014bf5640 1 -- 172.21.15.5:0/2678812154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6010141930 msgr2=0x7f60101e01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:47.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.997+0000 7f6014bf5640 1 --2- 172.21.15.5:0/2678812154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6010141930 0x7f60101e01a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f600400e930 tx=0x7f600400edf0 comp rx=0 tx=0).stop 2024-03-18T15:33:47.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.997+0000 7f6014bf5640 1 -- 172.21.15.5:0/2678812154 shutdown_connections 2024-03-18T15:33:47.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.997+0000 7f6014bf5640 1 --2- 172.21.15.5:0/2678812154 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5ff0067660 0x7f5ff0069b20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:47.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.997+0000 7f6014bf5640 1 --2- 172.21.15.5:0/2678812154 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f60101e4370 0x7f6010146980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:47.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.997+0000 7f6014bf5640 1 --2- 172.21.15.5:0/2678812154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6010141930 0x7f60101e01a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:47.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.997+0000 7f6014bf5640 1 --2- 172.21.15.5:0/2678812154 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f601006b4e0 0x7f601013f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:47.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.997+0000 7f6014bf5640 1 -- 172.21.15.5:0/2678812154 >> 172.21.15.5:0/2678812154 conn(0x7f60100b7a80 msgr2=0x7f60100b47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:47.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.997+0000 7f6014bf5640 1 -- 172.21.15.5:0/2678812154 shutdown_connections 2024-03-18T15:33:47.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:46.997+0000 7f6014bf5640 1 -- 172.21.15.5:0/2678812154 wait complete. 2024-03-18T15:33:47.016 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-03-18T15:33:46.886605+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":248028,"kb_used_data":2736,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754726692,"statfs":{"total":773094113280,"available":772840132608,"internally_reserved":0,"allocated":2801664,"data_stored":1715420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.930270"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.010963+0000","last_change":"2024-03-18T15:33:42.998484+0000","last_active":"2024-03-18T15:33:43.010963+0000","last_peered":"2024-03-18T15:33:43.010963+0000","last_clean":"2024-03-18T15:33:43.010963+0000","last_became_active":"2024-03-18T15:33:42.996054+0000","last_became_peered":"2024-03-18T15:33:42.996054+0000","last_unstale":"2024-03-18T15:33:43.010963+0000","last_undegraded":"2024-03-18T15:33:43.010963+0000","last_fullsized":"2024-03-18T15:33:43.010963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:08:08.722076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.001700+0000","last_change":"2024-03-18T15:33:42.997522+0000","last_active":"2024-03-18T15:33:43.001700+0000","last_peered":"2024-03-18T15:33:43.001700+0000","last_clean":"2024-03-18T15:33:43.001700+0000","last_became_active":"2024-03-18T15:33:42.997164+0000","last_became_peered":"2024-03-18T15:33:42.997164+0000","last_unstale":"2024-03-18T15:33:43.001700+0000","last_undegraded":"2024-03-18T15:33:43.001700+0000","last_fullsized":"2024-03-18T15:33:43.001700+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:43:43.250689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.002685+0000","last_change":"2024-03-18T15:33:43.001309+0000","last_active":"2024-03-18T15:33:43.002685+0000","last_peered":"2024-03-18T15:33:43.002685+0000","last_clean":"2024-03-18T15:33:43.002685+0000","last_became_active":"2024-03-18T15:33:43.000919+0000","last_became_peered":"2024-03-18T15:33:43.000919+0000","last_unstale":"2024-03-18T15:33:43.002685+0000","last_undegraded":"2024-03-18T15:33:43.002685+0000","last_fullsized":"2024-03-18T15:33:43.002685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:09:15.881784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.025684+0000","last_change":"2024-03-18T15:33:43.022575+0000","last_active":"2024-03-18T15:33:43.025684+0000","last_peered":"2024-03-18T15:33:43.025684+0000","last_clean":"2024-03-18T15:33:43.025684+0000","last_became_active":"2024-03-18T15:33:43.022221+0000","last_became_peered":"2024-03-18T15:33:43.022221+0000","last_unstale":"2024-03-18T15:33:43.025684+0000","last_undegraded":"2024-03-18T15:33:43.025684+0000","last_fullsized":"2024-03-18T15:33:43.025684+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:10:31.661870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.011405+0000","last_change":"2024-03-18T15:33:42.999245+0000","last_active":"2024-03-18T15:33:43.011405+0000","last_peered":"2024-03-18T15:33:43.011405+0000","last_clean":"2024-03-18T15:33:43.011405+0000","last_became_active":"2024-03-18T15:33:42.998951+0000","last_became_peered":"2024-03-18T15:33:42.998951+0000","last_unstale":"2024-03-18T15:33:43.011405+0000","last_undegraded":"2024-03-18T15:33:43.011405+0000","last_fullsized":"2024-03-18T15:33:43.011405+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:16:44.575392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.020131+0000","last_change":"2024-03-18T15:33:43.017400+0000","last_active":"2024-03-18T15:33:43.020131+0000","last_peered":"2024-03-18T15:33:43.020131+0000","last_clean":"2024-03-18T15:33:43.020131+0000","last_became_active":"2024-03-18T15:33:43.017074+0000","last_became_peered":"2024-03-18T15:33:43.017074+0000","last_unstale":"2024-03-18T15:33:43.020131+0000","last_undegraded":"2024-03-18T15:33:43.020131+0000","last_fullsized":"2024-03-18T15:33:43.020131+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:48:33.731710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.011020+0000","last_change":"2024-03-18T15:33:42.998214+0000","last_active":"2024-03-18T15:33:43.011020+0000","last_peered":"2024-03-18T15:33:43.011020+0000","last_clean":"2024-03-18T15:33:43.011020+0000","last_became_active":"2024-03-18T15:33:42.997922+0000","last_became_peered":"2024-03-18T15:33:42.997922+0000","last_unstale":"2024-03-18T15:33:43.011020+0000","last_undegraded":"2024-03-18T15:33:43.011020+0000","last_fullsized":"2024-03-18T15:33:43.011020+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:35:43.006937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.023025+0000","last_change":"2024-03-18T15:33:43.021183+0000","last_active":"2024-03-18T15:33:43.023025+0000","last_peered":"2024-03-18T15:33:43.023025+0000","last_clean":"2024-03-18T15:33:43.023025+0000","last_became_active":"2024-03-18T15:33:43.020848+0000","last_became_peered":"2024-03-18T15:33:43.020848+0000","last_unstale":"2024-03-18T15:33:43.023025+0000","last_undegraded":"2024-03-18T15:33:43.023025+0000","last_fullsized":"2024-03-18T15:33:43.023025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:46:08.824204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.025282+0000","last_change":"2024-03-18T15:33:43.018236+0000","last_active":"2024-03-18T15:33:43.025282+0000","last_peered":"2024-03-18T15:33:43.025282+0000","last_clean":"2024-03-18T15:33:43.025282+0000","last_became_active":"2024-03-18T15:33:43.017781+0000","last_became_peered":"2024-03-18T15:33:43.017781+0000","last_unstale":"2024-03-18T15:33:43.025282+0000","last_undegraded":"2024-03-18T15:33:43.025282+0000","last_fullsized":"2024-03-18T15:33:43.025282+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:29:18.287234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.002529+0000","last_change":"2024-03-18T15:33:43.020001+0000","last_active":"2024-03-18T15:33:44.002529+0000","last_peered":"2024-03-18T15:33:44.002529+0000","last_clean":"2024-03-18T15:33:44.002529+0000","last_became_active":"2024-03-18T15:33:43.019640+0000","last_became_peered":"2024-03-18T15:33:43.019640+0000","last_unstale":"2024-03-18T15:33:44.002529+0000","last_undegraded":"2024-03-18T15:33:44.002529+0000","last_fullsized":"2024-03-18T15:33:44.002529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:28:55.955807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.002450+0000","last_change":"2024-03-18T15:33:43.020077+0000","last_active":"2024-03-18T15:33:44.002450+0000","last_peered":"2024-03-18T15:33:44.002450+0000","last_clean":"2024-03-18T15:33:44.002450+0000","last_became_active":"2024-03-18T15:33:43.019703+0000","last_became_peered":"2024-03-18T15:33:43.019703+0000","last_unstale":"2024-03-18T15:33:44.002450+0000","last_undegraded":"2024-03-18T15:33:44.002450+0000","last_fullsized":"2024-03-18T15:33:44.002450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:10:31.661870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:42.998095+0000","last_change":"2024-03-18T15:33:42.988744+0000","last_active":"2024-03-18T15:33:42.998095+0000","last_peered":"2024-03-18T15:33:42.998095+0000","last_clean":"2024-03-18T15:33:42.998095+0000","last_became_active":"2024-03-18T15:33:42.988355+0000","last_became_peered":"2024-03-18T15:33:42.988355+0000","last_unstale":"2024-03-18T15:33:42.998095+0000","last_undegraded":"2024-03-18T15:33:42.998095+0000","last_fullsized":"2024-03-18T15:33:42.998095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:30:09.991617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.022233+0000","last_change":"2024-03-18T15:33:43.010027+0000","last_active":"2024-03-18T15:33:43.022233+0000","last_peered":"2024-03-18T15:33:43.022233+0000","last_clean":"2024-03-18T15:33:43.022233+0000","last_became_active":"2024-03-18T15:33:43.009401+0000","last_became_peered":"2024-03-18T15:33:43.009401+0000","last_unstale":"2024-03-18T15:33:43.022233+0000","last_undegraded":"2024-03-18T15:33:43.022233+0000","last_fullsized":"2024-03-18T15:33:43.022233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:43:42.420536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.002228+0000","last_change":"2024-03-18T15:33:42.999478+0000","last_active":"2024-03-18T15:33:43.002228+0000","last_peered":"2024-03-18T15:33:43.002228+0000","last_clean":"2024-03-18T15:33:43.002228+0000","last_became_active":"2024-03-18T15:33:42.998918+0000","last_became_peered":"2024-03-18T15:33:42.998918+0000","last_unstale":"2024-03-18T15:33:43.002228+0000","last_undegraded":"2024-03-18T15:33:43.002228+0000","last_fullsized":"2024-03-18T15:33:43.002228+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:41:02.566482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:42.994096+0000","last_change":"2024-03-18T15:33:42.983262+0000","last_active":"2024-03-18T15:33:42.994096+0000","last_peered":"2024-03-18T15:33:42.994096+0000","last_clean":"2024-03-18T15:33:42.994096+0000","last_became_active":"2024-03-18T15:33:42.982356+0000","last_became_peered":"2024-03-18T15:33:42.982356+0000","last_unstale":"2024-03-18T15:33:42.994096+0000","last_undegraded":"2024-03-18T15:33:42.994096+0000","last_fullsized":"2024-03-18T15:33:42.994096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T15:35:23.735580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.020122+0000","last_change":"2024-03-18T15:33:43.017589+0000","last_active":"2024-03-18T15:33:43.020122+0000","last_peered":"2024-03-18T15:33:43.020122+0000","last_clean":"2024-03-18T15:33:43.020122+0000","last_became_active":"2024-03-18T15:33:43.016991+0000","last_became_peered":"2024-03-18T15:33:43.016991+0000","last_unstale":"2024-03-18T15:33:43.020122+0000","last_undegraded":"2024-03-18T15:33:43.020122+0000","last_fullsized":"2024-03-18T15:33:43.020122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:59:23.040679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.018724+0000","last_change":"2024-03-18T15:33:43.015021+0000","last_active":"2024-03-18T15:33:43.018724+0000","last_peered":"2024-03-18T15:33:43.018724+0000","last_clean":"2024-03-18T15:33:43.018724+0000","last_became_active":"2024-03-18T15:33:43.012689+0000","last_became_peered":"2024-03-18T15:33:43.012689+0000","last_unstale":"2024-03-18T15:33:43.018724+0000","last_undegraded":"2024-03-18T15:33:43.018724+0000","last_fullsized":"2024-03-18T15:33:43.018724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:31:24.037103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.008664+0000","last_change":"2024-03-18T15:33:42.997254+0000","last_active":"2024-03-18T15:33:43.008664+0000","last_peered":"2024-03-18T15:33:43.008664+0000","last_clean":"2024-03-18T15:33:43.008664+0000","last_became_active":"2024-03-18T15:33:42.995704+0000","last_became_peered":"2024-03-18T15:33:42.995704+0000","last_unstale":"2024-03-18T15:33:43.008664+0000","last_undegraded":"2024-03-18T15:33:43.008664+0000","last_fullsized":"2024-03-18T15:33:43.008664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:39:47.487052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.011082+0000","last_change":"2024-03-18T15:33:42.998872+0000","last_active":"2024-03-18T15:33:43.011082+0000","last_peered":"2024-03-18T15:33:43.011082+0000","last_clean":"2024-03-18T15:33:43.011082+0000","last_became_active":"2024-03-18T15:33:42.998529+0000","last_became_peered":"2024-03-18T15:33:42.998529+0000","last_unstale":"2024-03-18T15:33:43.011082+0000","last_undegraded":"2024-03-18T15:33:43.011082+0000","last_fullsized":"2024-03-18T15:33:43.011082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:03:19.737667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.002213+0000","last_change":"2024-03-18T15:33:43.003843+0000","last_active":"2024-03-18T15:33:44.002213+0000","last_peered":"2024-03-18T15:33:44.002213+0000","last_clean":"2024-03-18T15:33:44.002213+0000","last_became_active":"2024-03-18T15:33:43.003481+0000","last_became_peered":"2024-03-18T15:33:43.003481+0000","last_unstale":"2024-03-18T15:33:44.002213+0000","last_undegraded":"2024-03-18T15:33:44.002213+0000","last_fullsized":"2024-03-18T15:33:44.002213+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:21:20.566649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.017405+0000","last_change":"2024-03-18T15:33:43.016217+0000","last_active":"2024-03-18T15:33:43.017405+0000","last_peered":"2024-03-18T15:33:43.017405+0000","last_clean":"2024-03-18T15:33:43.017405+0000","last_became_active":"2024-03-18T15:33:43.015511+0000","last_became_peered":"2024-03-18T15:33:43.015511+0000","last_unstale":"2024-03-18T15:33:43.017405+0000","last_undegraded":"2024-03-18T15:33:43.017405+0000","last_fullsized":"2024-03-18T15:33:43.017405+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:27:06.585839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.018826+0000","last_change":"2024-03-18T15:33:43.014558+0000","last_active":"2024-03-18T15:33:43.018826+0000","last_peered":"2024-03-18T15:33:43.018826+0000","last_clean":"2024-03-18T15:33:43.018826+0000","last_became_active":"2024-03-18T15:33:43.012745+0000","last_became_peered":"2024-03-18T15:33:43.012745+0000","last_unstale":"2024-03-18T15:33:43.018826+0000","last_undegraded":"2024-03-18T15:33:43.018826+0000","last_fullsized":"2024-03-18T15:33:43.018826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:34:52.856042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.022341+0000","last_change":"2024-03-18T15:33:43.010632+0000","last_active":"2024-03-18T15:33:43.022341+0000","last_peered":"2024-03-18T15:33:43.022341+0000","last_clean":"2024-03-18T15:33:43.022341+0000","last_became_active":"2024-03-18T15:33:43.010198+0000","last_became_peered":"2024-03-18T15:33:43.010198+0000","last_unstale":"2024-03-18T15:33:43.022341+0000","last_undegraded":"2024-03-18T15:33:43.022341+0000","last_fullsized":"2024-03-18T15:33:43.022341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:23:21.097164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.020802+0000","last_change":"2024-03-18T15:33:43.018016+0000","last_active":"2024-03-18T15:33:43.020802+0000","last_peered":"2024-03-18T15:33:43.020802+0000","last_clean":"2024-03-18T15:33:43.020802+0000","last_became_active":"2024-03-18T15:33:43.017655+0000","last_became_peered":"2024-03-18T15:33:43.017655+0000","last_unstale":"2024-03-18T15:33:43.020802+0000","last_undegraded":"2024-03-18T15:33:43.020802+0000","last_fullsized":"2024-03-18T15:33:43.020802+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:50:43.021257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.022722+0000","last_change":"2024-03-18T15:33:43.010776+0000","last_active":"2024-03-18T15:33:43.022722+0000","last_peered":"2024-03-18T15:33:43.022722+0000","last_clean":"2024-03-18T15:33:43.022722+0000","last_became_active":"2024-03-18T15:33:43.010222+0000","last_became_peered":"2024-03-18T15:33:43.010222+0000","last_unstale":"2024-03-18T15:33:43.022722+0000","last_undegraded":"2024-03-18T15:33:43.022722+0000","last_fullsized":"2024-03-18T15:33:43.022722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:14:43.243210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.002604+0000","last_change":"2024-03-18T15:33:43.011621+0000","last_active":"2024-03-18T15:33:44.002604+0000","last_peered":"2024-03-18T15:33:44.002604+0000","last_clean":"2024-03-18T15:33:44.002604+0000","last_became_active":"2024-03-18T15:33:43.011354+0000","last_became_peered":"2024-03-18T15:33:43.011354+0000","last_unstale":"2024-03-18T15:33:44.002604+0000","last_undegraded":"2024-03-18T15:33:44.002604+0000","last_fullsized":"2024-03-18T15:33:44.002604+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:50:28.968807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.012512+0000","last_change":"2024-03-18T15:33:43.011713+0000","last_active":"2024-03-18T15:33:43.012512+0000","last_peered":"2024-03-18T15:33:43.012512+0000","last_clean":"2024-03-18T15:33:43.012512+0000","last_became_active":"2024-03-18T15:33:43.011288+0000","last_became_peered":"2024-03-18T15:33:43.011288+0000","last_unstale":"2024-03-18T15:33:43.012512+0000","last_undegraded":"2024-03-18T15:33:43.012512+0000","last_fullsized":"2024-03-18T15:33:43.012512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:35:43.006937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.022946+0000","last_change":"2024-03-18T15:33:43.018468+0000","last_active":"2024-03-18T15:33:43.022946+0000","last_peered":"2024-03-18T15:33:43.022946+0000","last_clean":"2024-03-18T15:33:43.022946+0000","last_became_active":"2024-03-18T15:33:43.012834+0000","last_became_peered":"2024-03-18T15:33:43.012834+0000","last_unstale":"2024-03-18T15:33:43.022946+0000","last_undegraded":"2024-03-18T15:33:43.022946+0000","last_fullsized":"2024-03-18T15:33:43.022946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:29:34.725467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.017432+0000","last_change":"2024-03-18T15:33:43.016590+0000","last_active":"2024-03-18T15:33:43.017432+0000","last_peered":"2024-03-18T15:33:43.017432+0000","last_clean":"2024-03-18T15:33:43.017432+0000","last_became_active":"2024-03-18T15:33:43.015944+0000","last_became_peered":"2024-03-18T15:33:43.015944+0000","last_unstale":"2024-03-18T15:33:43.017432+0000","last_undegraded":"2024-03-18T15:33:43.017432+0000","last_fullsized":"2024-03-18T15:33:43.017432+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:13:38.420228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.018742+0000","last_change":"2024-03-18T15:33:43.014278+0000","last_active":"2024-03-18T15:33:43.018742+0000","last_peered":"2024-03-18T15:33:43.018742+0000","last_clean":"2024-03-18T15:33:43.018742+0000","last_became_active":"2024-03-18T15:33:43.012307+0000","last_became_peered":"2024-03-18T15:33:43.012307+0000","last_unstale":"2024-03-18T15:33:43.018742+0000","last_undegraded":"2024-03-18T15:33:43.018742+0000","last_fullsized":"2024-03-18T15:33:43.018742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:13:27.226842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.020049+0000","last_change":"2024-03-18T15:33:43.018853+0000","last_active":"2024-03-18T15:33:43.020049+0000","last_peered":"2024-03-18T15:33:43.020049+0000","last_clean":"2024-03-18T15:33:43.020049+0000","last_became_active":"2024-03-18T15:33:43.018496+0000","last_became_peered":"2024-03-18T15:33:43.018496+0000","last_unstale":"2024-03-18T15:33:43.020049+0000","last_undegraded":"2024-03-18T15:33:43.020049+0000","last_fullsized":"2024-03-18T15:33:43.020049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:27:06.585839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.002017+0000","last_change":"2024-03-18T15:33:43.000804+0000","last_active":"2024-03-18T15:33:44.002017+0000","last_peered":"2024-03-18T15:33:44.002017+0000","last_clean":"2024-03-18T15:33:44.002017+0000","last_became_active":"2024-03-18T15:33:43.000463+0000","last_became_peered":"2024-03-18T15:33:43.000463+0000","last_unstale":"2024-03-18T15:33:44.002017+0000","last_undegraded":"2024-03-18T15:33:44.002017+0000","last_fullsized":"2024-03-18T15:33:44.002017+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:57:58.192115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.005265+0000","last_change":"2024-03-18T15:33:42.995621+0000","last_active":"2024-03-18T15:33:43.005265+0000","last_peered":"2024-03-18T15:33:43.005265+0000","last_clean":"2024-03-18T15:33:43.005265+0000","last_became_active":"2024-03-18T15:33:42.995289+0000","last_became_peered":"2024-03-18T15:33:42.995289+0000","last_unstale":"2024-03-18T15:33:43.005265+0000","last_undegraded":"2024-03-18T15:33:43.005265+0000","last_fullsized":"2024-03-18T15:33:43.005265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:20:20.191051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.022435+0000","last_change":"2024-03-18T15:33:42.010459+0000","last_active":"2024-03-18T15:33:43.022435+0000","last_peered":"2024-03-18T15:33:43.022435+0000","last_clean":"2024-03-18T15:33:43.022435+0000","last_became_active":"2024-03-18T15:33:42.009978+0000","last_became_peered":"2024-03-18T15:33:42.009978+0000","last_unstale":"2024-03-18T15:33:43.022435+0000","last_undegraded":"2024-03-18T15:33:43.022435+0000","last_fullsized":"2024-03-18T15:33:43.022435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:20:19.179099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.016959+0000","last_change":"2024-03-18T15:33:43.015925+0000","last_active":"2024-03-18T15:33:43.016959+0000","last_peered":"2024-03-18T15:33:43.016959+0000","last_clean":"2024-03-18T15:33:43.016959+0000","last_became_active":"2024-03-18T15:33:43.014716+0000","last_became_peered":"2024-03-18T15:33:43.014716+0000","last_unstale":"2024-03-18T15:33:43.016959+0000","last_undegraded":"2024-03-18T15:33:43.016959+0000","last_fullsized":"2024-03-18T15:33:43.016959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:39:47.487052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.002101+0000","last_change":"2024-03-18T15:33:42.012969+0000","last_active":"2024-03-18T15:33:44.002101+0000","last_peered":"2024-03-18T15:33:44.002101+0000","last_clean":"2024-03-18T15:33:44.002101+0000","last_became_active":"2024-03-18T15:33:42.012416+0000","last_became_peered":"2024-03-18T15:33:42.012416+0000","last_unstale":"2024-03-18T15:33:44.002101+0000","last_undegraded":"2024-03-18T15:33:44.002101+0000","last_fullsized":"2024-03-18T15:33:44.002101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:47:53.438385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:42.983697+0000","last_change":"2024-03-18T15:33:41.996706+0000","last_active":"2024-03-18T15:33:42.983697+0000","last_peered":"2024-03-18T15:33:42.983697+0000","last_clean":"2024-03-18T15:33:42.983697+0000","last_became_active":"2024-03-18T15:33:41.996414+0000","last_became_peered":"2024-03-18T15:33:41.996414+0000","last_unstale":"2024-03-18T15:33:42.983697+0000","last_undegraded":"2024-03-18T15:33:42.983697+0000","last_fullsized":"2024-03-18T15:33:42.983697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:48:24.038250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.022207+0000","last_change":"2024-03-18T15:33:43.009392+0000","last_active":"2024-03-18T15:33:43.022207+0000","last_peered":"2024-03-18T15:33:43.022207+0000","last_clean":"2024-03-18T15:33:43.022207+0000","last_became_active":"2024-03-18T15:33:42.997447+0000","last_became_peered":"2024-03-18T15:33:42.997447+0000","last_unstale":"2024-03-18T15:33:43.022207+0000","last_undegraded":"2024-03-18T15:33:43.022207+0000","last_fullsized":"2024-03-18T15:33:43.022207+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:35:46.663732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.008520+0000","last_change":"2024-03-18T15:33:42.002322+0000","last_active":"2024-03-18T15:33:43.008520+0000","last_peered":"2024-03-18T15:33:43.008520+0000","last_clean":"2024-03-18T15:33:43.008520+0000","last_became_active":"2024-03-18T15:33:42.001646+0000","last_became_peered":"2024-03-18T15:33:42.001646+0000","last_unstale":"2024-03-18T15:33:43.008520+0000","last_undegraded":"2024-03-18T15:33:43.008520+0000","last_fullsized":"2024-03-18T15:33:43.008520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:39:46.475100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.004167+0000","last_change":"2024-03-18T15:33:42.998180+0000","last_active":"2024-03-18T15:33:43.004167+0000","last_peered":"2024-03-18T15:33:43.004167+0000","last_clean":"2024-03-18T15:33:43.004167+0000","last_became_active":"2024-03-18T15:33:42.997429+0000","last_became_peered":"2024-03-18T15:33:42.997429+0000","last_unstale":"2024-03-18T15:33:43.004167+0000","last_undegraded":"2024-03-18T15:33:43.004167+0000","last_fullsized":"2024-03-18T15:33:43.004167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:46:08.824204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.002488+0000","last_change":"2024-03-18T15:33:42.005707+0000","last_active":"2024-03-18T15:33:44.002488+0000","last_peered":"2024-03-18T15:33:44.002488+0000","last_clean":"2024-03-18T15:33:44.002488+0000","last_became_active":"2024-03-18T15:33:42.004925+0000","last_became_peered":"2024-03-18T15:33:42.004925+0000","last_unstale":"2024-03-18T15:33:44.002488+0000","last_undegraded":"2024-03-18T15:33:44.002488+0000","last_fullsized":"2024-03-18T15:33:44.002488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:39:46.475100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.018127+0000","last_change":"2024-03-18T15:33:43.012260+0000","last_active":"2024-03-18T15:33:43.018127+0000","last_peered":"2024-03-18T15:33:43.018127+0000","last_clean":"2024-03-18T15:33:43.018127+0000","last_became_active":"2024-03-18T15:33:43.011884+0000","last_became_peered":"2024-03-18T15:33:43.011884+0000","last_unstale":"2024-03-18T15:33:43.018127+0000","last_undegraded":"2024-03-18T15:33:43.018127+0000","last_fullsized":"2024-03-18T15:33:43.018127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:27:06.585839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:42.981446+0000","last_change":"2024-03-18T15:33:42.006507+0000","last_active":"2024-03-18T15:33:42.981446+0000","last_peered":"2024-03-18T15:33:42.981446+0000","last_clean":"2024-03-18T15:33:42.981446+0000","last_became_active":"2024-03-18T15:33:42.005178+0000","last_became_peered":"2024-03-18T15:33:42.005178+0000","last_unstale":"2024-03-18T15:33:42.981446+0000","last_undegraded":"2024-03-18T15:33:42.981446+0000","last_fullsized":"2024-03-18T15:33:42.981446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:09:14.869832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.021850+0000","last_change":"2024-03-18T15:33:43.021210+0000","last_active":"2024-03-18T15:33:43.021850+0000","last_peered":"2024-03-18T15:33:43.021850+0000","last_clean":"2024-03-18T15:33:43.021850+0000","last_became_active":"2024-03-18T15:33:43.020819+0000","last_became_peered":"2024-03-18T15:33:43.020819+0000","last_unstale":"2024-03-18T15:33:43.021850+0000","last_undegraded":"2024-03-18T15:33:43.021850+0000","last_fullsized":"2024-03-18T15:33:43.021850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:13:27.226842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.020694+0000","last_change":"2024-03-18T15:33:42.000763+0000","last_active":"2024-03-18T15:33:43.020694+0000","last_peered":"2024-03-18T15:33:43.020694+0000","last_clean":"2024-03-18T15:33:43.020694+0000","last_became_active":"2024-03-18T15:33:42.000392+0000","last_became_peered":"2024-03-18T15:33:42.000392+0000","last_unstale":"2024-03-18T15:33:43.020694+0000","last_undegraded":"2024-03-18T15:33:43.020694+0000","last_fullsized":"2024-03-18T15:33:43.020694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:43:41.408584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.009961+0000","last_change":"2024-03-18T15:33:43.003710+0000","last_active":"2024-03-18T15:33:43.009961+0000","last_peered":"2024-03-18T15:33:43.009961+0000","last_clean":"2024-03-18T15:33:43.009961+0000","last_became_active":"2024-03-18T15:33:43.003388+0000","last_became_peered":"2024-03-18T15:33:43.003388+0000","last_unstale":"2024-03-18T15:33:43.009961+0000","last_undegraded":"2024-03-18T15:33:43.009961+0000","last_fullsized":"2024-03-18T15:33:43.009961+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:48:25.050202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.002393+0000","last_change":"2024-03-18T15:33:41.993113+0000","last_active":"2024-03-18T15:33:44.002393+0000","last_peered":"2024-03-18T15:33:44.002393+0000","last_clean":"2024-03-18T15:33:44.002393+0000","last_became_active":"2024-03-18T15:33:41.992459+0000","last_became_peered":"2024-03-18T15:33:41.992459+0000","last_unstale":"2024-03-18T15:33:44.002393+0000","last_undegraded":"2024-03-18T15:33:44.002393+0000","last_fullsized":"2024-03-18T15:33:44.002393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:26:02.875362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.014709+0000","last_change":"2024-03-18T15:33:43.010684+0000","last_active":"2024-03-18T15:33:43.014709+0000","last_peered":"2024-03-18T15:33:43.014709+0000","last_clean":"2024-03-18T15:33:43.014709+0000","last_became_active":"2024-03-18T15:33:43.010021+0000","last_became_peered":"2024-03-18T15:33:43.010021+0000","last_unstale":"2024-03-18T15:33:43.014709+0000","last_undegraded":"2024-03-18T15:33:43.014709+0000","last_fullsized":"2024-03-18T15:33:43.014709+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:38:08.174152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.016892+0000","last_change":"2024-03-18T15:33:41.991859+0000","last_active":"2024-03-18T15:33:43.016892+0000","last_peered":"2024-03-18T15:33:43.016892+0000","last_clean":"2024-03-18T15:33:43.016892+0000","last_became_active":"2024-03-18T15:33:41.987429+0000","last_became_peered":"2024-03-18T15:33:41.987429+0000","last_unstale":"2024-03-18T15:33:43.016892+0000","last_undegraded":"2024-03-18T15:33:43.016892+0000","last_fullsized":"2024-03-18T15:33:43.016892+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:41:48.671691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.016342+0000","last_change":"2024-03-18T15:33:43.014886+0000","last_active":"2024-03-18T15:33:43.016342+0000","last_peered":"2024-03-18T15:33:43.016342+0000","last_clean":"2024-03-18T15:33:43.016342+0000","last_became_active":"2024-03-18T15:33:43.014430+0000","last_became_peered":"2024-03-18T15:33:43.014430+0000","last_unstale":"2024-03-18T15:33:43.016342+0000","last_undegraded":"2024-03-18T15:33:43.016342+0000","last_fullsized":"2024-03-18T15:33:43.016342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:47:54.450336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.007934+0000","last_change":"2024-03-18T15:33:42.000527+0000","last_active":"2024-03-18T15:33:43.007934+0000","last_peered":"2024-03-18T15:33:43.007934+0000","last_clean":"2024-03-18T15:33:43.007934+0000","last_became_active":"2024-03-18T15:33:41.999663+0000","last_became_peered":"2024-03-18T15:33:41.999663+0000","last_unstale":"2024-03-18T15:33:43.007934+0000","last_undegraded":"2024-03-18T15:33:43.007934+0000","last_fullsized":"2024-03-18T15:33:43.007934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:26:02.875362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.010993+0000","last_change":"2024-03-18T15:33:42.997482+0000","last_active":"2024-03-18T15:33:43.010993+0000","last_peered":"2024-03-18T15:33:43.010993+0000","last_clean":"2024-03-18T15:33:43.010993+0000","last_became_active":"2024-03-18T15:33:42.993674+0000","last_became_peered":"2024-03-18T15:33:42.993674+0000","last_unstale":"2024-03-18T15:33:43.010993+0000","last_undegraded":"2024-03-18T15:33:43.010993+0000","last_fullsized":"2024-03-18T15:33:43.010993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:46:08.824204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.001972+0000","last_change":"2024-03-18T15:33:41.990751+0000","last_active":"2024-03-18T15:33:44.001972+0000","last_peered":"2024-03-18T15:33:44.001972+0000","last_clean":"2024-03-18T15:33:44.001972+0000","last_became_active":"2024-03-18T15:33:41.989537+0000","last_became_peered":"2024-03-18T15:33:41.989537+0000","last_unstale":"2024-03-18T15:33:44.001972+0000","last_undegraded":"2024-03-18T15:33:44.001972+0000","last_fullsized":"2024-03-18T15:33:44.001972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:09:14.869832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.002581+0000","last_change":"2024-03-18T15:33:43.000356+0000","last_active":"2024-03-18T15:33:44.002581+0000","last_peered":"2024-03-18T15:33:44.002581+0000","last_clean":"2024-03-18T15:33:44.002581+0000","last_became_active":"2024-03-18T15:33:43.000031+0000","last_became_peered":"2024-03-18T15:33:43.000031+0000","last_unstale":"2024-03-18T15:33:44.002581+0000","last_undegraded":"2024-03-18T15:33:44.002581+0000","last_fullsized":"2024-03-18T15:33:44.002581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:24:37.779979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.011703+0000","last_change":"2024-03-18T15:33:41.976955+0000","last_active":"2024-03-18T15:33:43.011703+0000","last_peered":"2024-03-18T15:33:43.011703+0000","last_clean":"2024-03-18T15:33:43.011703+0000","last_became_active":"2024-03-18T15:33:41.976330+0000","last_became_peered":"2024-03-18T15:33:41.976330+0000","last_unstale":"2024-03-18T15:33:43.011703+0000","last_undegraded":"2024-03-18T15:33:43.011703+0000","last_fullsized":"2024-03-18T15:33:43.011703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:38:45.057586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.005397+0000","last_change":"2024-03-18T15:33:42.994227+0000","last_active":"2024-03-18T15:33:43.005397+0000","last_peered":"2024-03-18T15:33:43.005397+0000","last_clean":"2024-03-18T15:33:43.005397+0000","last_became_active":"2024-03-18T15:33:42.993870+0000","last_became_peered":"2024-03-18T15:33:42.993870+0000","last_unstale":"2024-03-18T15:33:43.005397+0000","last_undegraded":"2024-03-18T15:33:43.005397+0000","last_fullsized":"2024-03-18T15:33:43.005397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:43:43.250689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.015124+0000","last_change":"2024-03-18T15:33:41.987050+0000","last_active":"2024-03-18T15:33:43.015124+0000","last_peered":"2024-03-18T15:33:43.015124+0000","last_clean":"2024-03-18T15:33:43.015124+0000","last_became_active":"2024-03-18T15:33:41.986209+0000","last_became_peered":"2024-03-18T15:33:41.986209+0000","last_unstale":"2024-03-18T15:33:43.015124+0000","last_undegraded":"2024-03-18T15:33:43.015124+0000","last_fullsized":"2024-03-18T15:33:43.015124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:41:28.732114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.010873+0000","last_change":"2024-03-18T15:33:42.998067+0000","last_active":"2024-03-18T15:33:43.010873+0000","last_peered":"2024-03-18T15:33:43.010873+0000","last_clean":"2024-03-18T15:33:43.010873+0000","last_became_active":"2024-03-18T15:33:42.997755+0000","last_became_peered":"2024-03-18T15:33:42.997755+0000","last_unstale":"2024-03-18T15:33:43.010873+0000","last_undegraded":"2024-03-18T15:33:43.010873+0000","last_fullsized":"2024-03-18T15:33:43.010873+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:06:40.477304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.022995+0000","last_change":"2024-03-18T15:33:41.987009+0000","last_active":"2024-03-18T15:33:43.022995+0000","last_peered":"2024-03-18T15:33:43.022995+0000","last_clean":"2024-03-18T15:33:43.022995+0000","last_became_active":"2024-03-18T15:33:41.986376+0000","last_became_peered":"2024-03-18T15:33:41.986376+0000","last_unstale":"2024-03-18T15:33:43.022995+0000","last_undegraded":"2024-03-18T15:33:43.022995+0000","last_fullsized":"2024-03-18T15:33:43.022995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:26:02.875362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.014781+0000","last_change":"2024-03-18T15:33:43.007612+0000","last_active":"2024-03-18T15:33:43.014781+0000","last_peered":"2024-03-18T15:33:43.014781+0000","last_clean":"2024-03-18T15:33:43.014781+0000","last_became_active":"2024-03-18T15:33:43.006950+0000","last_became_peered":"2024-03-18T15:33:43.006950+0000","last_unstale":"2024-03-18T15:33:43.014781+0000","last_undegraded":"2024-03-18T15:33:43.014781+0000","last_fullsized":"2024-03-18T15:33:43.014781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:11:30.652938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.011210+0000","last_change":"2024-03-18T15:33:41.990256+0000","last_active":"2024-03-18T15:33:43.011210+0000","last_peered":"2024-03-18T15:33:43.011210+0000","last_clean":"2024-03-18T15:33:43.011210+0000","last_became_active":"2024-03-18T15:33:41.989715+0000","last_became_peered":"2024-03-18T15:33:41.989715+0000","last_unstale":"2024-03-18T15:33:43.011210+0000","last_undegraded":"2024-03-18T15:33:43.011210+0000","last_fullsized":"2024-03-18T15:33:43.011210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:23:20.085212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.025666+0000","last_change":"2024-03-18T15:33:43.022519+0000","last_active":"2024-03-18T15:33:43.025666+0000","last_peered":"2024-03-18T15:33:43.025666+0000","last_clean":"2024-03-18T15:33:43.025666+0000","last_became_active":"2024-03-18T15:33:43.022081+0000","last_became_peered":"2024-03-18T15:33:43.022081+0000","last_unstale":"2024-03-18T15:33:43.025666+0000","last_undegraded":"2024-03-18T15:33:43.025666+0000","last_fullsized":"2024-03-18T15:33:43.025666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:21:20.566649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:42.995557+0000","last_change":"2024-03-18T15:33:42.001343+0000","last_active":"2024-03-18T15:33:42.995557+0000","last_peered":"2024-03-18T15:33:42.995557+0000","last_clean":"2024-03-18T15:33:42.995557+0000","last_became_active":"2024-03-18T15:33:42.000948+0000","last_became_peered":"2024-03-18T15:33:42.000948+0000","last_unstale":"2024-03-18T15:33:42.995557+0000","last_undegraded":"2024-03-18T15:33:42.995557+0000","last_fullsized":"2024-03-18T15:33:42.995557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:47:53.438385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.022710+0000","last_change":"2024-03-18T15:33:43.015405+0000","last_active":"2024-03-18T15:33:43.022710+0000","last_peered":"2024-03-18T15:33:43.022710+0000","last_clean":"2024-03-18T15:33:43.022710+0000","last_became_active":"2024-03-18T15:33:43.015094+0000","last_became_peered":"2024-03-18T15:33:43.015094+0000","last_unstale":"2024-03-18T15:33:43.022710+0000","last_undegraded":"2024-03-18T15:33:43.022710+0000","last_fullsized":"2024-03-18T15:33:43.022710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:07:37.987066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":168,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:42.972906+0000","last_change":"2024-03-18T15:33:30.766315+0000","last_active":"2024-03-18T15:33:42.972906+0000","last_peered":"2024-03-18T15:33:42.972906+0000","last_clean":"2024-03-18T15:33:42.972906+0000","last_became_active":"2024-03-18T15:33:30.763939+0000","last_became_peered":"2024-03-18T15:33:30.763939+0000","last_unstale":"2024-03-18T15:33:42.972906+0000","last_undegraded":"2024-03-18T15:33:42.972906+0000","last_fullsized":"2024-03-18T15:33:42.972906+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:29.229911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:29.229911+0000","last_clean_scrub_stamp":"2024-03-18T15:33:29.229911+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:10:21.120371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:42.981172+0000","last_change":"2024-03-18T15:33:41.998201+0000","last_active":"2024-03-18T15:33:42.981172+0000","last_peered":"2024-03-18T15:33:42.981172+0000","last_clean":"2024-03-18T15:33:42.981172+0000","last_became_active":"2024-03-18T15:33:41.997079+0000","last_became_peered":"2024-03-18T15:33:41.997079+0000","last_unstale":"2024-03-18T15:33:42.981172+0000","last_undegraded":"2024-03-18T15:33:42.981172+0000","last_fullsized":"2024-03-18T15:33:42.981172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:38:45.057586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.022917+0000","last_change":"2024-03-18T15:33:43.019262+0000","last_active":"2024-03-18T15:33:43.022917+0000","last_peered":"2024-03-18T15:33:43.022917+0000","last_clean":"2024-03-18T15:33:43.022917+0000","last_became_active":"2024-03-18T15:33:43.018918+0000","last_became_peered":"2024-03-18T15:33:43.018918+0000","last_unstale":"2024-03-18T15:33:43.022917+0000","last_undegraded":"2024-03-18T15:33:43.022917+0000","last_fullsized":"2024-03-18T15:33:43.022917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:43:43.250689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:42.973219+0000","last_change":"2024-03-18T15:33:42.003947+0000","last_active":"2024-03-18T15:33:42.973219+0000","last_peered":"2024-03-18T15:33:42.973219+0000","last_clean":"2024-03-18T15:33:42.973219+0000","last_became_active":"2024-03-18T15:33:42.002250+0000","last_became_peered":"2024-03-18T15:33:42.002250+0000","last_unstale":"2024-03-18T15:33:42.973219+0000","last_undegraded":"2024-03-18T15:33:42.973219+0000","last_fullsized":"2024-03-18T15:33:42.973219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:07:04.089081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.016410+0000","last_change":"2024-03-18T15:33:43.015094+0000","last_active":"2024-03-18T15:33:43.016410+0000","last_peered":"2024-03-18T15:33:43.016410+0000","last_clean":"2024-03-18T15:33:43.016410+0000","last_became_active":"2024-03-18T15:33:43.014596+0000","last_became_peered":"2024-03-18T15:33:43.014596+0000","last_unstale":"2024-03-18T15:33:43.016410+0000","last_undegraded":"2024-03-18T15:33:43.016410+0000","last_fullsized":"2024-03-18T15:33:43.016410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:26:03.887314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:42.994962+0000","last_change":"2024-03-18T15:33:41.991149+0000","last_active":"2024-03-18T15:33:42.994962+0000","last_peered":"2024-03-18T15:33:42.994962+0000","last_clean":"2024-03-18T15:33:42.994962+0000","last_became_active":"2024-03-18T15:33:41.990059+0000","last_became_peered":"2024-03-18T15:33:41.990059+0000","last_unstale":"2024-03-18T15:33:42.994962+0000","last_undegraded":"2024-03-18T15:33:42.994962+0000","last_fullsized":"2024-03-18T15:33:42.994962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:09:14.869832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.006633+0000","last_change":"2024-03-18T15:33:43.000746+0000","last_active":"2024-03-18T15:33:43.006633+0000","last_peered":"2024-03-18T15:33:43.006633+0000","last_clean":"2024-03-18T15:33:43.006633+0000","last_became_active":"2024-03-18T15:33:42.999935+0000","last_became_peered":"2024-03-18T15:33:42.999935+0000","last_unstale":"2024-03-18T15:33:43.006633+0000","last_undegraded":"2024-03-18T15:33:43.006633+0000","last_fullsized":"2024-03-18T15:33:43.006633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:06:40.477304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:42.998144+0000","last_change":"2024-03-18T15:33:42.002409+0000","last_active":"2024-03-18T15:33:42.998144+0000","last_peered":"2024-03-18T15:33:42.998144+0000","last_clean":"2024-03-18T15:33:42.998144+0000","last_became_active":"2024-03-18T15:33:42.000616+0000","last_became_peered":"2024-03-18T15:33:42.000616+0000","last_unstale":"2024-03-18T15:33:42.998144+0000","last_undegraded":"2024-03-18T15:33:42.998144+0000","last_fullsized":"2024-03-18T15:33:42.998144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:47:53.438385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.010119+0000","last_change":"2024-03-18T15:33:43.005420+0000","last_active":"2024-03-18T15:33:43.010119+0000","last_peered":"2024-03-18T15:33:43.010119+0000","last_clean":"2024-03-18T15:33:43.010119+0000","last_became_active":"2024-03-18T15:33:43.004488+0000","last_became_peered":"2024-03-18T15:33:43.004488+0000","last_unstale":"2024-03-18T15:33:43.010119+0000","last_undegraded":"2024-03-18T15:33:43.010119+0000","last_fullsized":"2024-03-18T15:33:43.010119+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:46:34.352988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:42.970787+0000","last_change":"2024-03-18T15:33:41.990472+0000","last_active":"2024-03-18T15:33:42.970787+0000","last_peered":"2024-03-18T15:33:42.970787+0000","last_clean":"2024-03-18T15:33:42.970787+0000","last_became_active":"2024-03-18T15:33:41.989782+0000","last_became_peered":"2024-03-18T15:33:41.989782+0000","last_unstale":"2024-03-18T15:33:42.970787+0000","last_undegraded":"2024-03-18T15:33:42.970787+0000","last_fullsized":"2024-03-18T15:33:42.970787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:35:45.651780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.018367+0000","last_change":"2024-03-18T15:33:43.013011+0000","last_active":"2024-03-18T15:33:43.018367+0000","last_peered":"2024-03-18T15:33:43.018367+0000","last_clean":"2024-03-18T15:33:43.018367+0000","last_became_active":"2024-03-18T15:33:43.012334+0000","last_became_peered":"2024-03-18T15:33:43.012334+0000","last_unstale":"2024-03-18T15:33:43.018367+0000","last_undegraded":"2024-03-18T15:33:43.018367+0000","last_fullsized":"2024-03-18T15:33:43.018367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:13:38.420228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:42.979759+0000","last_change":"2024-03-18T15:33:42.011960+0000","last_active":"2024-03-18T15:33:42.979759+0000","last_peered":"2024-03-18T15:33:42.979759+0000","last_clean":"2024-03-18T15:33:42.979759+0000","last_became_active":"2024-03-18T15:33:42.010760+0000","last_became_peered":"2024-03-18T15:33:42.010760+0000","last_unstale":"2024-03-18T15:33:42.979759+0000","last_undegraded":"2024-03-18T15:33:42.979759+0000","last_fullsized":"2024-03-18T15:33:42.979759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:29:33.713515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:42.992570+0000","last_change":"2024-03-18T15:33:42.985474+0000","last_active":"2024-03-18T15:33:42.992570+0000","last_peered":"2024-03-18T15:33:42.992570+0000","last_clean":"2024-03-18T15:33:42.992570+0000","last_became_active":"2024-03-18T15:33:42.984659+0000","last_became_peered":"2024-03-18T15:33:42.984659+0000","last_unstale":"2024-03-18T15:33:42.992570+0000","last_undegraded":"2024-03-18T15:33:42.992570+0000","last_fullsized":"2024-03-18T15:33:42.992570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:29:34.725467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:42.971572+0000","last_change":"2024-03-18T15:33:41.994297+0000","last_active":"2024-03-18T15:33:42.971572+0000","last_peered":"2024-03-18T15:33:42.971572+0000","last_clean":"2024-03-18T15:33:42.971572+0000","last_became_active":"2024-03-18T15:33:41.993906+0000","last_became_peered":"2024-03-18T15:33:41.993906+0000","last_unstale":"2024-03-18T15:33:42.971572+0000","last_undegraded":"2024-03-18T15:33:42.971572+0000","last_fullsized":"2024-03-18T15:33:42.971572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:07:04.089081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.022744+0000","last_change":"2024-03-18T15:33:43.018039+0000","last_active":"2024-03-18T15:33:43.022744+0000","last_peered":"2024-03-18T15:33:43.022744+0000","last_clean":"2024-03-18T15:33:43.022744+0000","last_became_active":"2024-03-18T15:33:43.017680+0000","last_became_peered":"2024-03-18T15:33:43.017680+0000","last_unstale":"2024-03-18T15:33:43.022744+0000","last_undegraded":"2024-03-18T15:33:43.022744+0000","last_fullsized":"2024-03-18T15:33:43.022744+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T15:35:23.735580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.002591+0000","last_change":"2024-03-18T15:33:42.009920+0000","last_active":"2024-03-18T15:33:44.002591+0000","last_peered":"2024-03-18T15:33:44.002591+0000","last_clean":"2024-03-18T15:33:44.002591+0000","last_became_active":"2024-03-18T15:33:42.009394+0000","last_became_peered":"2024-03-18T15:33:42.009394+0000","last_unstale":"2024-03-18T15:33:44.002591+0000","last_undegraded":"2024-03-18T15:33:44.002591+0000","last_fullsized":"2024-03-18T15:33:44.002591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:27:05.573887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.004940+0000","last_change":"2024-03-18T15:33:42.999878+0000","last_active":"2024-03-18T15:33:43.004940+0000","last_peered":"2024-03-18T15:33:43.004940+0000","last_clean":"2024-03-18T15:33:43.004940+0000","last_became_active":"2024-03-18T15:33:42.999140+0000","last_became_peered":"2024-03-18T15:33:42.999140+0000","last_unstale":"2024-03-18T15:33:43.004940+0000","last_undegraded":"2024-03-18T15:33:43.004940+0000","last_fullsized":"2024-03-18T15:33:43.004940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:38:46.069538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:42.972728+0000","last_change":"2024-03-18T15:33:42.009110+0000","last_active":"2024-03-18T15:33:42.972728+0000","last_peered":"2024-03-18T15:33:42.972728+0000","last_clean":"2024-03-18T15:33:42.972728+0000","last_became_active":"2024-03-18T15:33:42.008110+0000","last_became_peered":"2024-03-18T15:33:42.008110+0000","last_unstale":"2024-03-18T15:33:42.972728+0000","last_undegraded":"2024-03-18T15:33:42.972728+0000","last_fullsized":"2024-03-18T15:33:42.972728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:23:20.085212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.011113+0000","last_change":"2024-03-18T15:33:43.006171+0000","last_active":"2024-03-18T15:33:43.011113+0000","last_peered":"2024-03-18T15:33:43.011113+0000","last_clean":"2024-03-18T15:33:43.011113+0000","last_became_active":"2024-03-18T15:33:43.005851+0000","last_became_peered":"2024-03-18T15:33:43.005851+0000","last_unstale":"2024-03-18T15:33:43.011113+0000","last_undegraded":"2024-03-18T15:33:43.011113+0000","last_fullsized":"2024-03-18T15:33:43.011113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:44:50.201644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:42.971290+0000","last_change":"2024-03-18T15:33:41.997380+0000","last_active":"2024-03-18T15:33:42.971290+0000","last_peered":"2024-03-18T15:33:42.971290+0000","last_clean":"2024-03-18T15:33:42.971290+0000","last_became_active":"2024-03-18T15:33:41.997101+0000","last_became_peered":"2024-03-18T15:33:41.997101+0000","last_unstale":"2024-03-18T15:33:42.971290+0000","last_undegraded":"2024-03-18T15:33:42.971290+0000","last_fullsized":"2024-03-18T15:33:42.971290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:33:11.480578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.022578+0000","last_change":"2024-03-18T15:33:43.012575+0000","last_active":"2024-03-18T15:33:43.022578+0000","last_peered":"2024-03-18T15:33:43.022578+0000","last_clean":"2024-03-18T15:33:43.022578+0000","last_became_active":"2024-03-18T15:33:42.996972+0000","last_became_peered":"2024-03-18T15:33:42.996972+0000","last_unstale":"2024-03-18T15:33:43.022578+0000","last_undegraded":"2024-03-18T15:33:43.022578+0000","last_fullsized":"2024-03-18T15:33:43.022578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:33:12.492530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:42.971759+0000","last_change":"2024-03-18T15:33:41.990440+0000","last_active":"2024-03-18T15:33:42.971759+0000","last_peered":"2024-03-18T15:33:42.971759+0000","last_clean":"2024-03-18T15:33:42.971759+0000","last_became_active":"2024-03-18T15:33:41.989808+0000","last_became_peered":"2024-03-18T15:33:41.989808+0000","last_unstale":"2024-03-18T15:33:42.971759+0000","last_undegraded":"2024-03-18T15:33:42.971759+0000","last_fullsized":"2024-03-18T15:33:42.971759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:14:42.231258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.022384+0000","last_change":"2024-03-18T15:33:43.012652+0000","last_active":"2024-03-18T15:33:43.022384+0000","last_peered":"2024-03-18T15:33:43.022384+0000","last_clean":"2024-03-18T15:33:43.022384+0000","last_became_active":"2024-03-18T15:33:43.010391+0000","last_became_peered":"2024-03-18T15:33:43.010391+0000","last_unstale":"2024-03-18T15:33:43.022384+0000","last_undegraded":"2024-03-18T15:33:43.022384+0000","last_fullsized":"2024-03-18T15:33:43.022384+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:07:05.101032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.011549+0000","last_change":"2024-03-18T15:33:42.013549+0000","last_active":"2024-03-18T15:33:43.011549+0000","last_peered":"2024-03-18T15:33:43.011549+0000","last_clean":"2024-03-18T15:33:43.011549+0000","last_became_active":"2024-03-18T15:33:42.012707+0000","last_became_peered":"2024-03-18T15:33:42.012707+0000","last_unstale":"2024-03-18T15:33:43.011549+0000","last_undegraded":"2024-03-18T15:33:43.011549+0000","last_fullsized":"2024-03-18T15:33:43.011549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:14:42.231258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.002049+0000","last_change":"2024-03-18T15:33:43.003144+0000","last_active":"2024-03-18T15:33:44.002049+0000","last_peered":"2024-03-18T15:33:44.002049+0000","last_clean":"2024-03-18T15:33:44.002049+0000","last_became_active":"2024-03-18T15:33:43.002771+0000","last_became_peered":"2024-03-18T15:33:43.002771+0000","last_unstale":"2024-03-18T15:33:44.002049+0000","last_undegraded":"2024-03-18T15:33:44.002049+0000","last_fullsized":"2024-03-18T15:33:44.002049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:50:43.021257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.018726+0000","last_change":"2024-03-18T15:33:43.014403+0000","last_active":"2024-03-18T15:33:43.018726+0000","last_peered":"2024-03-18T15:33:43.018726+0000","last_clean":"2024-03-18T15:33:43.018726+0000","last_became_active":"2024-03-18T15:33:43.012667+0000","last_became_peered":"2024-03-18T15:33:43.012667+0000","last_unstale":"2024-03-18T15:33:43.018726+0000","last_undegraded":"2024-03-18T15:33:43.018726+0000","last_fullsized":"2024-03-18T15:33:43.018726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:43:00.095618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.019002+0000","last_change":"2024-03-18T15:33:42.009430+0000","last_active":"2024-03-18T15:33:43.019002+0000","last_peered":"2024-03-18T15:33:43.019002+0000","last_clean":"2024-03-18T15:33:43.019002+0000","last_became_active":"2024-03-18T15:33:42.008331+0000","last_became_peered":"2024-03-18T15:33:42.008331+0000","last_unstale":"2024-03-18T15:33:43.019002+0000","last_undegraded":"2024-03-18T15:33:43.019002+0000","last_fullsized":"2024-03-18T15:33:43.019002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:33:11.480578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.020201+0000","last_change":"2024-03-18T15:33:43.017670+0000","last_active":"2024-03-18T15:33:43.020201+0000","last_peered":"2024-03-18T15:33:43.020201+0000","last_clean":"2024-03-18T15:33:43.020201+0000","last_became_active":"2024-03-18T15:33:43.017297+0000","last_became_peered":"2024-03-18T15:33:43.017297+0000","last_unstale":"2024-03-18T15:33:43.020201+0000","last_undegraded":"2024-03-18T15:33:43.020201+0000","last_fullsized":"2024-03-18T15:33:43.020201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:57:58.192115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.011284+0000","last_change":"2024-03-18T15:33:42.009132+0000","last_active":"2024-03-18T15:33:43.011284+0000","last_peered":"2024-03-18T15:33:43.011284+0000","last_clean":"2024-03-18T15:33:43.011284+0000","last_became_active":"2024-03-18T15:33:42.008343+0000","last_became_peered":"2024-03-18T15:33:42.008343+0000","last_unstale":"2024-03-18T15:33:43.011284+0000","last_undegraded":"2024-03-18T15:33:43.011284+0000","last_fullsized":"2024-03-18T15:33:43.011284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T15:35:22.723629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.021872+0000","last_change":"2024-03-18T15:33:43.021277+0000","last_active":"2024-03-18T15:33:43.021872+0000","last_peered":"2024-03-18T15:33:43.021872+0000","last_clean":"2024-03-18T15:33:43.021872+0000","last_became_active":"2024-03-18T15:33:43.020857+0000","last_became_peered":"2024-03-18T15:33:43.020857+0000","last_unstale":"2024-03-18T15:33:43.021872+0000","last_undegraded":"2024-03-18T15:33:43.021872+0000","last_fullsized":"2024-03-18T15:33:43.021872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:43:00.095618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.022483+0000","last_change":"2024-03-18T15:33:41.992048+0000","last_active":"2024-03-18T15:33:43.022483+0000","last_peered":"2024-03-18T15:33:43.022483+0000","last_clean":"2024-03-18T15:33:43.022483+0000","last_became_active":"2024-03-18T15:33:41.989055+0000","last_became_peered":"2024-03-18T15:33:41.989055+0000","last_unstale":"2024-03-18T15:33:43.022483+0000","last_undegraded":"2024-03-18T15:33:43.022483+0000","last_fullsized":"2024-03-18T15:33:43.022483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:46:33.341037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.019817+0000","last_change":"2024-03-18T15:33:42.996569+0000","last_active":"2024-03-18T15:33:43.019817+0000","last_peered":"2024-03-18T15:33:43.019817+0000","last_clean":"2024-03-18T15:33:43.019817+0000","last_became_active":"2024-03-18T15:33:42.996234+0000","last_became_peered":"2024-03-18T15:33:42.996234+0000","last_unstale":"2024-03-18T15:33:43.019817+0000","last_undegraded":"2024-03-18T15:33:43.019817+0000","last_fullsized":"2024-03-18T15:33:43.019817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:44:50.201644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.019674+0000","last_change":"2024-03-18T15:33:42.010405+0000","last_active":"2024-03-18T15:33:43.019674+0000","last_peered":"2024-03-18T15:33:43.019674+0000","last_clean":"2024-03-18T15:33:43.019674+0000","last_became_active":"2024-03-18T15:33:42.009883+0000","last_became_peered":"2024-03-18T15:33:42.009883+0000","last_unstale":"2024-03-18T15:33:43.019674+0000","last_undegraded":"2024-03-18T15:33:43.019674+0000","last_fullsized":"2024-03-18T15:33:43.019674+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:10:06.802810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":75067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":200704,"data_stored":67245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":794624,"data_stored":658105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":184320,"data_stored":60380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35548,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336292,"statfs":{"total":96636764160,"available":96600363008,"internally_reserved":0,"allocated":208896,"data_stored":68767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35524,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336316,"statfs":{"total":96636764160,"available":96600387584,"internally_reserved":0,"allocated":184320,"data_stored":60380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":204800,"data_stored":67315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36124,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335716,"statfs":{"total":96636764160,"available":96599773184,"internally_reserved":0,"allocated":802816,"data_stored":658161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-18T15:33:47.020 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-18T15:33:47.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.177+0000 7f6696e60640 1 Processor -- start 2024-03-18T15:33:47.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.181+0000 7f6696e60640 1 -- start start 2024-03-18T15:33:47.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.181+0000 7f6696e60640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f66900ba040 0x7f66900b5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:47.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.181+0000 7f6696e60640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66900ba420 con 0x7f6690141700 2024-03-18T15:33:47.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.181+0000 7f6696e60640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66900b44c0 con 0x7f669014aa40 2024-03-18T15:33:47.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.181+0000 7f6696e60640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f66900b4fa0 con 0x7f66900ba040 2024-03-18T15:33:47.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.181+0000 7f66953fd640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f6690141700 0x7f6690151dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:57212/0 (socket says 172.21.15.5:57212) 2024-03-18T15:33:47.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.181+0000 7f66953fd640 1 -- 172.21.15.5:0/1993261656 learned_addr learned my addr 172.21.15.5:0/1993261656 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:33:47.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.181+0000 7f6694bfc640 1 --2- 172.21.15.5:0/1993261656 >> v2:172.21.15.80:3300/0 conn(0x7f66900ba040 0x7f66900b5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:47.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.181+0000 7f668f7fe640 1 -- 172.21.15.5:0/1993261656 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2098018318 0 0) 0x7f66900ba420 con 0x7f6690141700 2024-03-18T15:33:47.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.181+0000 7f668f7fe640 1 -- 172.21.15.5:0/1993261656 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6678003530 con 0x7f6690141700 2024-03-18T15:33:47.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.181+0000 7f668f7fe640 1 -- 172.21.15.5:0/1993261656 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3416305396 0 0) 0x7f6678003530 con 0x7f6690141700 2024-03-18T15:33:47.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.181+0000 7f668f7fe640 1 -- 172.21.15.5:0/1993261656 >> v2:172.21.15.80:3300/0 conn(0x7f66900ba040 msgr2=0x7f66900b5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:47.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.181+0000 7f668f7fe640 1 --2- 172.21.15.5:0/1993261656 >> v2:172.21.15.80:3300/0 conn(0x7f66900ba040 0x7f66900b5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:47.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.181+0000 7f668f7fe640 1 -- 172.21.15.5:0/1993261656 >> v1:172.21.15.80:6789/0 conn(0x7f669014aa40 legacy=0x7f6690140ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:47.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.181+0000 7f668f7fe640 1 -- 172.21.15.5:0/1993261656 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66900b47a0 con 0x7f6690141700 2024-03-18T15:33:47.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.181+0000 7f668f7fe640 1 -- 172.21.15.5:0/1993261656 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3837549003 0 0) 0x7f6688001a00 con 0x7f6690141700 2024-03-18T15:33:47.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.181+0000 7f668f7fe640 1 -- 172.21.15.5:0/1993261656 >> v1:172.21.15.5:6789/0 conn(0x7f6690141700 legacy=0x7f6690151dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:47.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.181+0000 7f668f7fe640 1 --2- 172.21.15.5:0/1993261656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6678003aa0 0x7f6678003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:47.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.181+0000 7f668f7fe640 1 --2- 172.21.15.5:0/1993261656 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6678004810 0x7f6678004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:47.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.181+0000 7f668f7fe640 1 --2- 172.21.15.5:0/1993261656 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f66780053d0 0x7f667800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:47.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.181+0000 7f668f7fe640 1 -- 172.21.15.5:0/1993261656 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f667800c260 con 0x7f6678003aa0 2024-03-18T15:33:47.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.181+0000 7f668f7fe640 1 -- 172.21.15.5:0/1993261656 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f667800c410 con 0x7f66780053d0 2024-03-18T15:33:47.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.181+0000 7f668f7fe640 1 -- 172.21.15.5:0/1993261656 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f667800c610 con 0x7f6678004810 2024-03-18T15:33:47.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.181+0000 7f6694bfc640 1 --2- 172.21.15.5:0/1993261656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6678003aa0 0x7f6678003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:47.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.181+0000 7f66953fd640 1 --2- 172.21.15.5:0/1993261656 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f66780053d0 0x7f667800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:47.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.181+0000 7f6694bfc640 1 -- 172.21.15.5:0/1993261656 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6678004810 msgr2=0x7f6678004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:47.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.181+0000 7f668ffff640 1 --2- 172.21.15.5:0/1993261656 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6678004810 0x7f6678004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:47.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.181+0000 7f6694bfc640 1 --2- 172.21.15.5:0/1993261656 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6678004810 0x7f6678004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:47.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.181+0000 7f6694bfc640 1 -- 172.21.15.5:0/1993261656 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f66780053d0 msgr2=0x7f667800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:47.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.181+0000 7f6694bfc640 1 --2- 172.21.15.5:0/1993261656 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f66780053d0 0x7f667800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:47.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.181+0000 7f6694bfc640 1 -- 172.21.15.5:0/1993261656 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6678003530 con 0x7f6678003aa0 2024-03-18T15:33:47.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.181+0000 7f66953fd640 1 --2- 172.21.15.5:0/1993261656 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f66780053d0 0x7f667800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:33:47.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.181+0000 7f66953fd640 1 -- 172.21.15.5:0/1993261656 reap_dead start 2024-03-18T15:33:47.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.185+0000 7f6694bfc640 1 --2- 172.21.15.5:0/1993261656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6678003aa0 0x7f6678003ec0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6680009ae0 tx=0x7f668001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=263156d549b2c30b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:47.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.185+0000 7f668f7fe640 1 -- 172.21.15.5:0/1993261656 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66800096b0 con 0x7f6678003aa0 2024-03-18T15:33:47.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.185+0000 7f668f7fe640 1 -- 172.21.15.5:0/1993261656 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66800a6450 con 0x7f6678003aa0 2024-03-18T15:33:47.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.185+0000 7f668f7fe640 1 -- 172.21.15.5:0/1993261656 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66800a6840 con 0x7f6678003aa0 2024-03-18T15:33:47.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.185+0000 7f6696e60640 1 -- 172.21.15.5:0/1993261656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6678003aa0 msgr2=0x7f6678003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:47.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.185+0000 7f6696e60640 1 --2- 172.21.15.5:0/1993261656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6678003aa0 0x7f6678003ec0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6680009ae0 tx=0x7f668001ca70 comp rx=0 tx=0).stop 2024-03-18T15:33:47.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.185+0000 7f6696e60640 1 -- 172.21.15.5:0/1993261656 shutdown_connections 2024-03-18T15:33:47.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.185+0000 7f6696e60640 1 --2- 172.21.15.5:0/1993261656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6678003aa0 0x7f6678003ec0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:47.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.185+0000 7f6696e60640 1 -- 172.21.15.5:0/1993261656 >> 172.21.15.5:0/1993261656 conn(0x7f66900673c0 msgr2=0x7f66900677d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:47.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.185+0000 7f6696e60640 1 -- 172.21.15.5:0/1993261656 shutdown_connections 2024-03-18T15:33:47.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.185+0000 7f6696e60640 1 -- 172.21.15.5:0/1993261656 wait complete. 2024-03-18T15:33:47.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.185+0000 7f6696e60640 1 Processor -- start 2024-03-18T15:33:47.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.185+0000 7f6696e60640 1 -- start start 2024-03-18T15:33:47.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.185+0000 7f6696e60640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6678003aa0 0x7f669013b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:47.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.185+0000 7f6696e60640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f669013bec0 0x7f669013c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:47.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.185+0000 7f6696e60640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f669013cb10 0x7f669013cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:47.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.185+0000 7f6696e60640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f66900b95b0 con 0x7f669013cb10 2024-03-18T15:33:47.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.185+0000 7f6696e60640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f66900b9430 con 0x7f669013bec0 2024-03-18T15:33:47.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.185+0000 7f6696e60640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f66900b57e0 con 0x7f6678003aa0 2024-03-18T15:33:47.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.185+0000 7f66953fd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f669013cb10 0x7f669013cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:47.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.185+0000 7f66953fd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f669013cb10 0x7f669013cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38346/0 (socket says 172.21.15.5:38346) 2024-03-18T15:33:47.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.185+0000 7f66953fd640 1 -- 172.21.15.5:0/2769555638 learned_addr learned my addr 172.21.15.5:0/2769555638 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:47.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.185+0000 7f668ffff640 1 --2- 172.21.15.5:0/2769555638 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f669013bec0 0x7f669013c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:47.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.185+0000 7f6694bfc640 1 --2- 172.21.15.5:0/2769555638 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6678003aa0 0x7f669013b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:47.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.185+0000 7f66953fd640 1 -- 172.21.15.5:0/2769555638 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6678003aa0 msgr2=0x7f669013b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:47.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.185+0000 7f66953fd640 1 --2- 172.21.15.5:0/2769555638 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6678003aa0 0x7f669013b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:47.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.185+0000 7f66953fd640 1 -- 172.21.15.5:0/2769555638 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f669013bec0 msgr2=0x7f669013c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:47.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.185+0000 7f66953fd640 1 --2- 172.21.15.5:0/2769555638 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f669013bec0 0x7f669013c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:47.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.185+0000 7f66953fd640 1 -- 172.21.15.5:0/2769555638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f669013abb0 con 0x7f669013cb10 2024-03-18T15:33:47.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.185+0000 7f66953fd640 1 --2- 172.21.15.5:0/2769555638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f669013cb10 0x7f669013cf90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f66900b67c0 tx=0x7f668800df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:47.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.189+0000 7f668d7fa640 1 -- 172.21.15.5:0/2769555638 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668800b7a0 con 0x7f669013cb10 2024-03-18T15:33:47.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.189+0000 7f668d7fa640 1 -- 172.21.15.5:0/2769555638 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f668800b940 con 0x7f669013cb10 2024-03-18T15:33:47.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.189+0000 7f668d7fa640 1 -- 172.21.15.5:0/2769555638 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668800bd30 con 0x7f669013cb10 2024-03-18T15:33:47.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.189+0000 7f6696e60640 1 -- 172.21.15.5:0/2769555638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66901e02e0 con 0x7f669013cb10 2024-03-18T15:33:47.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.189+0000 7f6696e60640 1 -- 172.21.15.5:0/2769555638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66901e0880 con 0x7f669013cb10 2024-03-18T15:33:47.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.189+0000 7f6696e60640 1 -- 172.21.15.5:0/2769555638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f669004f550 con 0x7f669013cb10 2024-03-18T15:33:47.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.189+0000 7f668d7fa640 1 -- 172.21.15.5:0/2769555638 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f668801b020 con 0x7f669013cb10 2024-03-18T15:33:47.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.189+0000 7f668d7fa640 1 --2- 172.21.15.5:0/2769555638 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f667c067690 0x7f667c069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:47.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.189+0000 7f6694bfc640 1 --2- 172.21.15.5:0/2769555638 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f667c067690 0x7f667c069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:47.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.189+0000 7f668d7fa640 1 -- 172.21.15.5:0/2769555638 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f6688085820 con 0x7f669013cb10 2024-03-18T15:33:47.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.189+0000 7f6694bfc640 1 --2- 172.21.15.5:0/2769555638 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f667c067690 0x7f667c069b50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f66800ad000 tx=0x7f66800046d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:47.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.193+0000 7f668d7fa640 1 -- 172.21.15.5:0/2769555638 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f6688056490 con 0x7f669013cb10 2024-03-18T15:33:47.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.349+0000 7f6696e60640 1 -- 172.21.15.5:0/2769555638 --> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f66900ba420 con 0x7f667c067690 2024-03-18T15:33:47.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.365+0000 7f668d7fa640 1 -- 172.21.15.5:0/2769555638 <== mgr.4110 v2:172.21.15.5:6832/3362034860 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251883 (secure 0 0 0) 0x7f66900ba420 con 0x7f667c067690 2024-03-18T15:33:47.371 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:33:47.372 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-03-18T15:33:47.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.369+0000 7f6696e60640 1 -- 172.21.15.5:0/2769555638 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f667c067690 msgr2=0x7f667c069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:47.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.369+0000 7f6696e60640 1 --2- 172.21.15.5:0/2769555638 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f667c067690 0x7f667c069b50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f66800ad000 tx=0x7f66800046d0 comp rx=0 tx=0).stop 2024-03-18T15:33:47.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.369+0000 7f6696e60640 1 -- 172.21.15.5:0/2769555638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f669013cb10 msgr2=0x7f669013cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:47.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.369+0000 7f6696e60640 1 --2- 172.21.15.5:0/2769555638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f669013cb10 0x7f669013cf90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f66900b67c0 tx=0x7f668800df70 comp rx=0 tx=0).stop 2024-03-18T15:33:47.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.369+0000 7f6696e60640 1 -- 172.21.15.5:0/2769555638 shutdown_connections 2024-03-18T15:33:47.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.369+0000 7f6696e60640 1 --2- 172.21.15.5:0/2769555638 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f667c067690 0x7f667c069b50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:47.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.369+0000 7f6696e60640 1 --2- 172.21.15.5:0/2769555638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f669013cb10 0x7f669013cf90 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:47.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.369+0000 7f6696e60640 1 --2- 172.21.15.5:0/2769555638 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f669013bec0 0x7f669013c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:47.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.369+0000 7f6696e60640 1 --2- 172.21.15.5:0/2769555638 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6678003aa0 0x7f669013b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:47.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.369+0000 7f6696e60640 1 -- 172.21.15.5:0/2769555638 >> 172.21.15.5:0/2769555638 conn(0x7f66900673c0 msgr2=0x7f6690139630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:47.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.369+0000 7f6696e60640 1 -- 172.21.15.5:0/2769555638 shutdown_connections 2024-03-18T15:33:47.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.369+0000 7f6696e60640 1 -- 172.21.15.5:0/2769555638 wait complete. 2024-03-18T15:33:47.388 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-03-18T15:33:46.886605+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":248028,"kb_used_data":2736,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754726692,"statfs":{"total":773094113280,"available":772840132608,"internally_reserved":0,"allocated":2801664,"data_stored":1715420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.930270"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.010963+0000","last_change":"2024-03-18T15:33:42.998484+0000","last_active":"2024-03-18T15:33:43.010963+0000","last_peered":"2024-03-18T15:33:43.010963+0000","last_clean":"2024-03-18T15:33:43.010963+0000","last_became_active":"2024-03-18T15:33:42.996054+0000","last_became_peered":"2024-03-18T15:33:42.996054+0000","last_unstale":"2024-03-18T15:33:43.010963+0000","last_undegraded":"2024-03-18T15:33:43.010963+0000","last_fullsized":"2024-03-18T15:33:43.010963+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:08:08.722076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.001700+0000","last_change":"2024-03-18T15:33:42.997522+0000","last_active":"2024-03-18T15:33:43.001700+0000","last_peered":"2024-03-18T15:33:43.001700+0000","last_clean":"2024-03-18T15:33:43.001700+0000","last_became_active":"2024-03-18T15:33:42.997164+0000","last_became_peered":"2024-03-18T15:33:42.997164+0000","last_unstale":"2024-03-18T15:33:43.001700+0000","last_undegraded":"2024-03-18T15:33:43.001700+0000","last_fullsized":"2024-03-18T15:33:43.001700+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:43:43.250689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.002685+0000","last_change":"2024-03-18T15:33:43.001309+0000","last_active":"2024-03-18T15:33:43.002685+0000","last_peered":"2024-03-18T15:33:43.002685+0000","last_clean":"2024-03-18T15:33:43.002685+0000","last_became_active":"2024-03-18T15:33:43.000919+0000","last_became_peered":"2024-03-18T15:33:43.000919+0000","last_unstale":"2024-03-18T15:33:43.002685+0000","last_undegraded":"2024-03-18T15:33:43.002685+0000","last_fullsized":"2024-03-18T15:33:43.002685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:09:15.881784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.025684+0000","last_change":"2024-03-18T15:33:43.022575+0000","last_active":"2024-03-18T15:33:43.025684+0000","last_peered":"2024-03-18T15:33:43.025684+0000","last_clean":"2024-03-18T15:33:43.025684+0000","last_became_active":"2024-03-18T15:33:43.022221+0000","last_became_peered":"2024-03-18T15:33:43.022221+0000","last_unstale":"2024-03-18T15:33:43.025684+0000","last_undegraded":"2024-03-18T15:33:43.025684+0000","last_fullsized":"2024-03-18T15:33:43.025684+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:10:31.661870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.011405+0000","last_change":"2024-03-18T15:33:42.999245+0000","last_active":"2024-03-18T15:33:43.011405+0000","last_peered":"2024-03-18T15:33:43.011405+0000","last_clean":"2024-03-18T15:33:43.011405+0000","last_became_active":"2024-03-18T15:33:42.998951+0000","last_became_peered":"2024-03-18T15:33:42.998951+0000","last_unstale":"2024-03-18T15:33:43.011405+0000","last_undegraded":"2024-03-18T15:33:43.011405+0000","last_fullsized":"2024-03-18T15:33:43.011405+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:16:44.575392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.020131+0000","last_change":"2024-03-18T15:33:43.017400+0000","last_active":"2024-03-18T15:33:43.020131+0000","last_peered":"2024-03-18T15:33:43.020131+0000","last_clean":"2024-03-18T15:33:43.020131+0000","last_became_active":"2024-03-18T15:33:43.017074+0000","last_became_peered":"2024-03-18T15:33:43.017074+0000","last_unstale":"2024-03-18T15:33:43.020131+0000","last_undegraded":"2024-03-18T15:33:43.020131+0000","last_fullsized":"2024-03-18T15:33:43.020131+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:48:33.731710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.011020+0000","last_change":"2024-03-18T15:33:42.998214+0000","last_active":"2024-03-18T15:33:43.011020+0000","last_peered":"2024-03-18T15:33:43.011020+0000","last_clean":"2024-03-18T15:33:43.011020+0000","last_became_active":"2024-03-18T15:33:42.997922+0000","last_became_peered":"2024-03-18T15:33:42.997922+0000","last_unstale":"2024-03-18T15:33:43.011020+0000","last_undegraded":"2024-03-18T15:33:43.011020+0000","last_fullsized":"2024-03-18T15:33:43.011020+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:35:43.006937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.023025+0000","last_change":"2024-03-18T15:33:43.021183+0000","last_active":"2024-03-18T15:33:43.023025+0000","last_peered":"2024-03-18T15:33:43.023025+0000","last_clean":"2024-03-18T15:33:43.023025+0000","last_became_active":"2024-03-18T15:33:43.020848+0000","last_became_peered":"2024-03-18T15:33:43.020848+0000","last_unstale":"2024-03-18T15:33:43.023025+0000","last_undegraded":"2024-03-18T15:33:43.023025+0000","last_fullsized":"2024-03-18T15:33:43.023025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:46:08.824204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.025282+0000","last_change":"2024-03-18T15:33:43.018236+0000","last_active":"2024-03-18T15:33:43.025282+0000","last_peered":"2024-03-18T15:33:43.025282+0000","last_clean":"2024-03-18T15:33:43.025282+0000","last_became_active":"2024-03-18T15:33:43.017781+0000","last_became_peered":"2024-03-18T15:33:43.017781+0000","last_unstale":"2024-03-18T15:33:43.025282+0000","last_undegraded":"2024-03-18T15:33:43.025282+0000","last_fullsized":"2024-03-18T15:33:43.025282+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:29:18.287234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.002529+0000","last_change":"2024-03-18T15:33:43.020001+0000","last_active":"2024-03-18T15:33:44.002529+0000","last_peered":"2024-03-18T15:33:44.002529+0000","last_clean":"2024-03-18T15:33:44.002529+0000","last_became_active":"2024-03-18T15:33:43.019640+0000","last_became_peered":"2024-03-18T15:33:43.019640+0000","last_unstale":"2024-03-18T15:33:44.002529+0000","last_undegraded":"2024-03-18T15:33:44.002529+0000","last_fullsized":"2024-03-18T15:33:44.002529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:28:55.955807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.002450+0000","last_change":"2024-03-18T15:33:43.020077+0000","last_active":"2024-03-18T15:33:44.002450+0000","last_peered":"2024-03-18T15:33:44.002450+0000","last_clean":"2024-03-18T15:33:44.002450+0000","last_became_active":"2024-03-18T15:33:43.019703+0000","last_became_peered":"2024-03-18T15:33:43.019703+0000","last_unstale":"2024-03-18T15:33:44.002450+0000","last_undegraded":"2024-03-18T15:33:44.002450+0000","last_fullsized":"2024-03-18T15:33:44.002450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:10:31.661870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:42.998095+0000","last_change":"2024-03-18T15:33:42.988744+0000","last_active":"2024-03-18T15:33:42.998095+0000","last_peered":"2024-03-18T15:33:42.998095+0000","last_clean":"2024-03-18T15:33:42.998095+0000","last_became_active":"2024-03-18T15:33:42.988355+0000","last_became_peered":"2024-03-18T15:33:42.988355+0000","last_unstale":"2024-03-18T15:33:42.998095+0000","last_undegraded":"2024-03-18T15:33:42.998095+0000","last_fullsized":"2024-03-18T15:33:42.998095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:30:09.991617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.022233+0000","last_change":"2024-03-18T15:33:43.010027+0000","last_active":"2024-03-18T15:33:43.022233+0000","last_peered":"2024-03-18T15:33:43.022233+0000","last_clean":"2024-03-18T15:33:43.022233+0000","last_became_active":"2024-03-18T15:33:43.009401+0000","last_became_peered":"2024-03-18T15:33:43.009401+0000","last_unstale":"2024-03-18T15:33:43.022233+0000","last_undegraded":"2024-03-18T15:33:43.022233+0000","last_fullsized":"2024-03-18T15:33:43.022233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:43:42.420536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.002228+0000","last_change":"2024-03-18T15:33:42.999478+0000","last_active":"2024-03-18T15:33:43.002228+0000","last_peered":"2024-03-18T15:33:43.002228+0000","last_clean":"2024-03-18T15:33:43.002228+0000","last_became_active":"2024-03-18T15:33:42.998918+0000","last_became_peered":"2024-03-18T15:33:42.998918+0000","last_unstale":"2024-03-18T15:33:43.002228+0000","last_undegraded":"2024-03-18T15:33:43.002228+0000","last_fullsized":"2024-03-18T15:33:43.002228+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:41:02.566482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:42.994096+0000","last_change":"2024-03-18T15:33:42.983262+0000","last_active":"2024-03-18T15:33:42.994096+0000","last_peered":"2024-03-18T15:33:42.994096+0000","last_clean":"2024-03-18T15:33:42.994096+0000","last_became_active":"2024-03-18T15:33:42.982356+0000","last_became_peered":"2024-03-18T15:33:42.982356+0000","last_unstale":"2024-03-18T15:33:42.994096+0000","last_undegraded":"2024-03-18T15:33:42.994096+0000","last_fullsized":"2024-03-18T15:33:42.994096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T15:35:23.735580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.020122+0000","last_change":"2024-03-18T15:33:43.017589+0000","last_active":"2024-03-18T15:33:43.020122+0000","last_peered":"2024-03-18T15:33:43.020122+0000","last_clean":"2024-03-18T15:33:43.020122+0000","last_became_active":"2024-03-18T15:33:43.016991+0000","last_became_peered":"2024-03-18T15:33:43.016991+0000","last_unstale":"2024-03-18T15:33:43.020122+0000","last_undegraded":"2024-03-18T15:33:43.020122+0000","last_fullsized":"2024-03-18T15:33:43.020122+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:59:23.040679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.018724+0000","last_change":"2024-03-18T15:33:43.015021+0000","last_active":"2024-03-18T15:33:43.018724+0000","last_peered":"2024-03-18T15:33:43.018724+0000","last_clean":"2024-03-18T15:33:43.018724+0000","last_became_active":"2024-03-18T15:33:43.012689+0000","last_became_peered":"2024-03-18T15:33:43.012689+0000","last_unstale":"2024-03-18T15:33:43.018724+0000","last_undegraded":"2024-03-18T15:33:43.018724+0000","last_fullsized":"2024-03-18T15:33:43.018724+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:31:24.037103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.008664+0000","last_change":"2024-03-18T15:33:42.997254+0000","last_active":"2024-03-18T15:33:43.008664+0000","last_peered":"2024-03-18T15:33:43.008664+0000","last_clean":"2024-03-18T15:33:43.008664+0000","last_became_active":"2024-03-18T15:33:42.995704+0000","last_became_peered":"2024-03-18T15:33:42.995704+0000","last_unstale":"2024-03-18T15:33:43.008664+0000","last_undegraded":"2024-03-18T15:33:43.008664+0000","last_fullsized":"2024-03-18T15:33:43.008664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:39:47.487052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.011082+0000","last_change":"2024-03-18T15:33:42.998872+0000","last_active":"2024-03-18T15:33:43.011082+0000","last_peered":"2024-03-18T15:33:43.011082+0000","last_clean":"2024-03-18T15:33:43.011082+0000","last_became_active":"2024-03-18T15:33:42.998529+0000","last_became_peered":"2024-03-18T15:33:42.998529+0000","last_unstale":"2024-03-18T15:33:43.011082+0000","last_undegraded":"2024-03-18T15:33:43.011082+0000","last_fullsized":"2024-03-18T15:33:43.011082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:03:19.737667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.002213+0000","last_change":"2024-03-18T15:33:43.003843+0000","last_active":"2024-03-18T15:33:44.002213+0000","last_peered":"2024-03-18T15:33:44.002213+0000","last_clean":"2024-03-18T15:33:44.002213+0000","last_became_active":"2024-03-18T15:33:43.003481+0000","last_became_peered":"2024-03-18T15:33:43.003481+0000","last_unstale":"2024-03-18T15:33:44.002213+0000","last_undegraded":"2024-03-18T15:33:44.002213+0000","last_fullsized":"2024-03-18T15:33:44.002213+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:21:20.566649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.017405+0000","last_change":"2024-03-18T15:33:43.016217+0000","last_active":"2024-03-18T15:33:43.017405+0000","last_peered":"2024-03-18T15:33:43.017405+0000","last_clean":"2024-03-18T15:33:43.017405+0000","last_became_active":"2024-03-18T15:33:43.015511+0000","last_became_peered":"2024-03-18T15:33:43.015511+0000","last_unstale":"2024-03-18T15:33:43.017405+0000","last_undegraded":"2024-03-18T15:33:43.017405+0000","last_fullsized":"2024-03-18T15:33:43.017405+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:27:06.585839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.018826+0000","last_change":"2024-03-18T15:33:43.014558+0000","last_active":"2024-03-18T15:33:43.018826+0000","last_peered":"2024-03-18T15:33:43.018826+0000","last_clean":"2024-03-18T15:33:43.018826+0000","last_became_active":"2024-03-18T15:33:43.012745+0000","last_became_peered":"2024-03-18T15:33:43.012745+0000","last_unstale":"2024-03-18T15:33:43.018826+0000","last_undegraded":"2024-03-18T15:33:43.018826+0000","last_fullsized":"2024-03-18T15:33:43.018826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:34:52.856042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.022341+0000","last_change":"2024-03-18T15:33:43.010632+0000","last_active":"2024-03-18T15:33:43.022341+0000","last_peered":"2024-03-18T15:33:43.022341+0000","last_clean":"2024-03-18T15:33:43.022341+0000","last_became_active":"2024-03-18T15:33:43.010198+0000","last_became_peered":"2024-03-18T15:33:43.010198+0000","last_unstale":"2024-03-18T15:33:43.022341+0000","last_undegraded":"2024-03-18T15:33:43.022341+0000","last_fullsized":"2024-03-18T15:33:43.022341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:23:21.097164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.020802+0000","last_change":"2024-03-18T15:33:43.018016+0000","last_active":"2024-03-18T15:33:43.020802+0000","last_peered":"2024-03-18T15:33:43.020802+0000","last_clean":"2024-03-18T15:33:43.020802+0000","last_became_active":"2024-03-18T15:33:43.017655+0000","last_became_peered":"2024-03-18T15:33:43.017655+0000","last_unstale":"2024-03-18T15:33:43.020802+0000","last_undegraded":"2024-03-18T15:33:43.020802+0000","last_fullsized":"2024-03-18T15:33:43.020802+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:50:43.021257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.022722+0000","last_change":"2024-03-18T15:33:43.010776+0000","last_active":"2024-03-18T15:33:43.022722+0000","last_peered":"2024-03-18T15:33:43.022722+0000","last_clean":"2024-03-18T15:33:43.022722+0000","last_became_active":"2024-03-18T15:33:43.010222+0000","last_became_peered":"2024-03-18T15:33:43.010222+0000","last_unstale":"2024-03-18T15:33:43.022722+0000","last_undegraded":"2024-03-18T15:33:43.022722+0000","last_fullsized":"2024-03-18T15:33:43.022722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:14:43.243210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.002604+0000","last_change":"2024-03-18T15:33:43.011621+0000","last_active":"2024-03-18T15:33:44.002604+0000","last_peered":"2024-03-18T15:33:44.002604+0000","last_clean":"2024-03-18T15:33:44.002604+0000","last_became_active":"2024-03-18T15:33:43.011354+0000","last_became_peered":"2024-03-18T15:33:43.011354+0000","last_unstale":"2024-03-18T15:33:44.002604+0000","last_undegraded":"2024-03-18T15:33:44.002604+0000","last_fullsized":"2024-03-18T15:33:44.002604+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:50:28.968807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.012512+0000","last_change":"2024-03-18T15:33:43.011713+0000","last_active":"2024-03-18T15:33:43.012512+0000","last_peered":"2024-03-18T15:33:43.012512+0000","last_clean":"2024-03-18T15:33:43.012512+0000","last_became_active":"2024-03-18T15:33:43.011288+0000","last_became_peered":"2024-03-18T15:33:43.011288+0000","last_unstale":"2024-03-18T15:33:43.012512+0000","last_undegraded":"2024-03-18T15:33:43.012512+0000","last_fullsized":"2024-03-18T15:33:43.012512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:35:43.006937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.022946+0000","last_change":"2024-03-18T15:33:43.018468+0000","last_active":"2024-03-18T15:33:43.022946+0000","last_peered":"2024-03-18T15:33:43.022946+0000","last_clean":"2024-03-18T15:33:43.022946+0000","last_became_active":"2024-03-18T15:33:43.012834+0000","last_became_peered":"2024-03-18T15:33:43.012834+0000","last_unstale":"2024-03-18T15:33:43.022946+0000","last_undegraded":"2024-03-18T15:33:43.022946+0000","last_fullsized":"2024-03-18T15:33:43.022946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:29:34.725467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.017432+0000","last_change":"2024-03-18T15:33:43.016590+0000","last_active":"2024-03-18T15:33:43.017432+0000","last_peered":"2024-03-18T15:33:43.017432+0000","last_clean":"2024-03-18T15:33:43.017432+0000","last_became_active":"2024-03-18T15:33:43.015944+0000","last_became_peered":"2024-03-18T15:33:43.015944+0000","last_unstale":"2024-03-18T15:33:43.017432+0000","last_undegraded":"2024-03-18T15:33:43.017432+0000","last_fullsized":"2024-03-18T15:33:43.017432+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:13:38.420228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.018742+0000","last_change":"2024-03-18T15:33:43.014278+0000","last_active":"2024-03-18T15:33:43.018742+0000","last_peered":"2024-03-18T15:33:43.018742+0000","last_clean":"2024-03-18T15:33:43.018742+0000","last_became_active":"2024-03-18T15:33:43.012307+0000","last_became_peered":"2024-03-18T15:33:43.012307+0000","last_unstale":"2024-03-18T15:33:43.018742+0000","last_undegraded":"2024-03-18T15:33:43.018742+0000","last_fullsized":"2024-03-18T15:33:43.018742+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:13:27.226842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.020049+0000","last_change":"2024-03-18T15:33:43.018853+0000","last_active":"2024-03-18T15:33:43.020049+0000","last_peered":"2024-03-18T15:33:43.020049+0000","last_clean":"2024-03-18T15:33:43.020049+0000","last_became_active":"2024-03-18T15:33:43.018496+0000","last_became_peered":"2024-03-18T15:33:43.018496+0000","last_unstale":"2024-03-18T15:33:43.020049+0000","last_undegraded":"2024-03-18T15:33:43.020049+0000","last_fullsized":"2024-03-18T15:33:43.020049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:27:06.585839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.002017+0000","last_change":"2024-03-18T15:33:43.000804+0000","last_active":"2024-03-18T15:33:44.002017+0000","last_peered":"2024-03-18T15:33:44.002017+0000","last_clean":"2024-03-18T15:33:44.002017+0000","last_became_active":"2024-03-18T15:33:43.000463+0000","last_became_peered":"2024-03-18T15:33:43.000463+0000","last_unstale":"2024-03-18T15:33:44.002017+0000","last_undegraded":"2024-03-18T15:33:44.002017+0000","last_fullsized":"2024-03-18T15:33:44.002017+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:57:58.192115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.005265+0000","last_change":"2024-03-18T15:33:42.995621+0000","last_active":"2024-03-18T15:33:43.005265+0000","last_peered":"2024-03-18T15:33:43.005265+0000","last_clean":"2024-03-18T15:33:43.005265+0000","last_became_active":"2024-03-18T15:33:42.995289+0000","last_became_peered":"2024-03-18T15:33:42.995289+0000","last_unstale":"2024-03-18T15:33:43.005265+0000","last_undegraded":"2024-03-18T15:33:43.005265+0000","last_fullsized":"2024-03-18T15:33:43.005265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:20:20.191051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.022435+0000","last_change":"2024-03-18T15:33:42.010459+0000","last_active":"2024-03-18T15:33:43.022435+0000","last_peered":"2024-03-18T15:33:43.022435+0000","last_clean":"2024-03-18T15:33:43.022435+0000","last_became_active":"2024-03-18T15:33:42.009978+0000","last_became_peered":"2024-03-18T15:33:42.009978+0000","last_unstale":"2024-03-18T15:33:43.022435+0000","last_undegraded":"2024-03-18T15:33:43.022435+0000","last_fullsized":"2024-03-18T15:33:43.022435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:20:19.179099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.016959+0000","last_change":"2024-03-18T15:33:43.015925+0000","last_active":"2024-03-18T15:33:43.016959+0000","last_peered":"2024-03-18T15:33:43.016959+0000","last_clean":"2024-03-18T15:33:43.016959+0000","last_became_active":"2024-03-18T15:33:43.014716+0000","last_became_peered":"2024-03-18T15:33:43.014716+0000","last_unstale":"2024-03-18T15:33:43.016959+0000","last_undegraded":"2024-03-18T15:33:43.016959+0000","last_fullsized":"2024-03-18T15:33:43.016959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:39:47.487052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.002101+0000","last_change":"2024-03-18T15:33:42.012969+0000","last_active":"2024-03-18T15:33:44.002101+0000","last_peered":"2024-03-18T15:33:44.002101+0000","last_clean":"2024-03-18T15:33:44.002101+0000","last_became_active":"2024-03-18T15:33:42.012416+0000","last_became_peered":"2024-03-18T15:33:42.012416+0000","last_unstale":"2024-03-18T15:33:44.002101+0000","last_undegraded":"2024-03-18T15:33:44.002101+0000","last_fullsized":"2024-03-18T15:33:44.002101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:47:53.438385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:42.983697+0000","last_change":"2024-03-18T15:33:41.996706+0000","last_active":"2024-03-18T15:33:42.983697+0000","last_peered":"2024-03-18T15:33:42.983697+0000","last_clean":"2024-03-18T15:33:42.983697+0000","last_became_active":"2024-03-18T15:33:41.996414+0000","last_became_peered":"2024-03-18T15:33:41.996414+0000","last_unstale":"2024-03-18T15:33:42.983697+0000","last_undegraded":"2024-03-18T15:33:42.983697+0000","last_fullsized":"2024-03-18T15:33:42.983697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:48:24.038250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.022207+0000","last_change":"2024-03-18T15:33:43.009392+0000","last_active":"2024-03-18T15:33:43.022207+0000","last_peered":"2024-03-18T15:33:43.022207+0000","last_clean":"2024-03-18T15:33:43.022207+0000","last_became_active":"2024-03-18T15:33:42.997447+0000","last_became_peered":"2024-03-18T15:33:42.997447+0000","last_unstale":"2024-03-18T15:33:43.022207+0000","last_undegraded":"2024-03-18T15:33:43.022207+0000","last_fullsized":"2024-03-18T15:33:43.022207+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:35:46.663732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.008520+0000","last_change":"2024-03-18T15:33:42.002322+0000","last_active":"2024-03-18T15:33:43.008520+0000","last_peered":"2024-03-18T15:33:43.008520+0000","last_clean":"2024-03-18T15:33:43.008520+0000","last_became_active":"2024-03-18T15:33:42.001646+0000","last_became_peered":"2024-03-18T15:33:42.001646+0000","last_unstale":"2024-03-18T15:33:43.008520+0000","last_undegraded":"2024-03-18T15:33:43.008520+0000","last_fullsized":"2024-03-18T15:33:43.008520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:39:46.475100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.004167+0000","last_change":"2024-03-18T15:33:42.998180+0000","last_active":"2024-03-18T15:33:43.004167+0000","last_peered":"2024-03-18T15:33:43.004167+0000","last_clean":"2024-03-18T15:33:43.004167+0000","last_became_active":"2024-03-18T15:33:42.997429+0000","last_became_peered":"2024-03-18T15:33:42.997429+0000","last_unstale":"2024-03-18T15:33:43.004167+0000","last_undegraded":"2024-03-18T15:33:43.004167+0000","last_fullsized":"2024-03-18T15:33:43.004167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:46:08.824204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.002488+0000","last_change":"2024-03-18T15:33:42.005707+0000","last_active":"2024-03-18T15:33:44.002488+0000","last_peered":"2024-03-18T15:33:44.002488+0000","last_clean":"2024-03-18T15:33:44.002488+0000","last_became_active":"2024-03-18T15:33:42.004925+0000","last_became_peered":"2024-03-18T15:33:42.004925+0000","last_unstale":"2024-03-18T15:33:44.002488+0000","last_undegraded":"2024-03-18T15:33:44.002488+0000","last_fullsized":"2024-03-18T15:33:44.002488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:39:46.475100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.018127+0000","last_change":"2024-03-18T15:33:43.012260+0000","last_active":"2024-03-18T15:33:43.018127+0000","last_peered":"2024-03-18T15:33:43.018127+0000","last_clean":"2024-03-18T15:33:43.018127+0000","last_became_active":"2024-03-18T15:33:43.011884+0000","last_became_peered":"2024-03-18T15:33:43.011884+0000","last_unstale":"2024-03-18T15:33:43.018127+0000","last_undegraded":"2024-03-18T15:33:43.018127+0000","last_fullsized":"2024-03-18T15:33:43.018127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:27:06.585839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:42.981446+0000","last_change":"2024-03-18T15:33:42.006507+0000","last_active":"2024-03-18T15:33:42.981446+0000","last_peered":"2024-03-18T15:33:42.981446+0000","last_clean":"2024-03-18T15:33:42.981446+0000","last_became_active":"2024-03-18T15:33:42.005178+0000","last_became_peered":"2024-03-18T15:33:42.005178+0000","last_unstale":"2024-03-18T15:33:42.981446+0000","last_undegraded":"2024-03-18T15:33:42.981446+0000","last_fullsized":"2024-03-18T15:33:42.981446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:09:14.869832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.021850+0000","last_change":"2024-03-18T15:33:43.021210+0000","last_active":"2024-03-18T15:33:43.021850+0000","last_peered":"2024-03-18T15:33:43.021850+0000","last_clean":"2024-03-18T15:33:43.021850+0000","last_became_active":"2024-03-18T15:33:43.020819+0000","last_became_peered":"2024-03-18T15:33:43.020819+0000","last_unstale":"2024-03-18T15:33:43.021850+0000","last_undegraded":"2024-03-18T15:33:43.021850+0000","last_fullsized":"2024-03-18T15:33:43.021850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:13:27.226842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.020694+0000","last_change":"2024-03-18T15:33:42.000763+0000","last_active":"2024-03-18T15:33:43.020694+0000","last_peered":"2024-03-18T15:33:43.020694+0000","last_clean":"2024-03-18T15:33:43.020694+0000","last_became_active":"2024-03-18T15:33:42.000392+0000","last_became_peered":"2024-03-18T15:33:42.000392+0000","last_unstale":"2024-03-18T15:33:43.020694+0000","last_undegraded":"2024-03-18T15:33:43.020694+0000","last_fullsized":"2024-03-18T15:33:43.020694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:43:41.408584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.009961+0000","last_change":"2024-03-18T15:33:43.003710+0000","last_active":"2024-03-18T15:33:43.009961+0000","last_peered":"2024-03-18T15:33:43.009961+0000","last_clean":"2024-03-18T15:33:43.009961+0000","last_became_active":"2024-03-18T15:33:43.003388+0000","last_became_peered":"2024-03-18T15:33:43.003388+0000","last_unstale":"2024-03-18T15:33:43.009961+0000","last_undegraded":"2024-03-18T15:33:43.009961+0000","last_fullsized":"2024-03-18T15:33:43.009961+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:48:25.050202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.002393+0000","last_change":"2024-03-18T15:33:41.993113+0000","last_active":"2024-03-18T15:33:44.002393+0000","last_peered":"2024-03-18T15:33:44.002393+0000","last_clean":"2024-03-18T15:33:44.002393+0000","last_became_active":"2024-03-18T15:33:41.992459+0000","last_became_peered":"2024-03-18T15:33:41.992459+0000","last_unstale":"2024-03-18T15:33:44.002393+0000","last_undegraded":"2024-03-18T15:33:44.002393+0000","last_fullsized":"2024-03-18T15:33:44.002393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:26:02.875362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.014709+0000","last_change":"2024-03-18T15:33:43.010684+0000","last_active":"2024-03-18T15:33:43.014709+0000","last_peered":"2024-03-18T15:33:43.014709+0000","last_clean":"2024-03-18T15:33:43.014709+0000","last_became_active":"2024-03-18T15:33:43.010021+0000","last_became_peered":"2024-03-18T15:33:43.010021+0000","last_unstale":"2024-03-18T15:33:43.014709+0000","last_undegraded":"2024-03-18T15:33:43.014709+0000","last_fullsized":"2024-03-18T15:33:43.014709+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:38:08.174152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.016892+0000","last_change":"2024-03-18T15:33:41.991859+0000","last_active":"2024-03-18T15:33:43.016892+0000","last_peered":"2024-03-18T15:33:43.016892+0000","last_clean":"2024-03-18T15:33:43.016892+0000","last_became_active":"2024-03-18T15:33:41.987429+0000","last_became_peered":"2024-03-18T15:33:41.987429+0000","last_unstale":"2024-03-18T15:33:43.016892+0000","last_undegraded":"2024-03-18T15:33:43.016892+0000","last_fullsized":"2024-03-18T15:33:43.016892+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:41:48.671691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.016342+0000","last_change":"2024-03-18T15:33:43.014886+0000","last_active":"2024-03-18T15:33:43.016342+0000","last_peered":"2024-03-18T15:33:43.016342+0000","last_clean":"2024-03-18T15:33:43.016342+0000","last_became_active":"2024-03-18T15:33:43.014430+0000","last_became_peered":"2024-03-18T15:33:43.014430+0000","last_unstale":"2024-03-18T15:33:43.016342+0000","last_undegraded":"2024-03-18T15:33:43.016342+0000","last_fullsized":"2024-03-18T15:33:43.016342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:47:54.450336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.007934+0000","last_change":"2024-03-18T15:33:42.000527+0000","last_active":"2024-03-18T15:33:43.007934+0000","last_peered":"2024-03-18T15:33:43.007934+0000","last_clean":"2024-03-18T15:33:43.007934+0000","last_became_active":"2024-03-18T15:33:41.999663+0000","last_became_peered":"2024-03-18T15:33:41.999663+0000","last_unstale":"2024-03-18T15:33:43.007934+0000","last_undegraded":"2024-03-18T15:33:43.007934+0000","last_fullsized":"2024-03-18T15:33:43.007934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:26:02.875362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.010993+0000","last_change":"2024-03-18T15:33:42.997482+0000","last_active":"2024-03-18T15:33:43.010993+0000","last_peered":"2024-03-18T15:33:43.010993+0000","last_clean":"2024-03-18T15:33:43.010993+0000","last_became_active":"2024-03-18T15:33:42.993674+0000","last_became_peered":"2024-03-18T15:33:42.993674+0000","last_unstale":"2024-03-18T15:33:43.010993+0000","last_undegraded":"2024-03-18T15:33:43.010993+0000","last_fullsized":"2024-03-18T15:33:43.010993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:46:08.824204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.001972+0000","last_change":"2024-03-18T15:33:41.990751+0000","last_active":"2024-03-18T15:33:44.001972+0000","last_peered":"2024-03-18T15:33:44.001972+0000","last_clean":"2024-03-18T15:33:44.001972+0000","last_became_active":"2024-03-18T15:33:41.989537+0000","last_became_peered":"2024-03-18T15:33:41.989537+0000","last_unstale":"2024-03-18T15:33:44.001972+0000","last_undegraded":"2024-03-18T15:33:44.001972+0000","last_fullsized":"2024-03-18T15:33:44.001972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:09:14.869832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.002581+0000","last_change":"2024-03-18T15:33:43.000356+0000","last_active":"2024-03-18T15:33:44.002581+0000","last_peered":"2024-03-18T15:33:44.002581+0000","last_clean":"2024-03-18T15:33:44.002581+0000","last_became_active":"2024-03-18T15:33:43.000031+0000","last_became_peered":"2024-03-18T15:33:43.000031+0000","last_unstale":"2024-03-18T15:33:44.002581+0000","last_undegraded":"2024-03-18T15:33:44.002581+0000","last_fullsized":"2024-03-18T15:33:44.002581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:24:37.779979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.011703+0000","last_change":"2024-03-18T15:33:41.976955+0000","last_active":"2024-03-18T15:33:43.011703+0000","last_peered":"2024-03-18T15:33:43.011703+0000","last_clean":"2024-03-18T15:33:43.011703+0000","last_became_active":"2024-03-18T15:33:41.976330+0000","last_became_peered":"2024-03-18T15:33:41.976330+0000","last_unstale":"2024-03-18T15:33:43.011703+0000","last_undegraded":"2024-03-18T15:33:43.011703+0000","last_fullsized":"2024-03-18T15:33:43.011703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:38:45.057586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.005397+0000","last_change":"2024-03-18T15:33:42.994227+0000","last_active":"2024-03-18T15:33:43.005397+0000","last_peered":"2024-03-18T15:33:43.005397+0000","last_clean":"2024-03-18T15:33:43.005397+0000","last_became_active":"2024-03-18T15:33:42.993870+0000","last_became_peered":"2024-03-18T15:33:42.993870+0000","last_unstale":"2024-03-18T15:33:43.005397+0000","last_undegraded":"2024-03-18T15:33:43.005397+0000","last_fullsized":"2024-03-18T15:33:43.005397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:43:43.250689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.015124+0000","last_change":"2024-03-18T15:33:41.987050+0000","last_active":"2024-03-18T15:33:43.015124+0000","last_peered":"2024-03-18T15:33:43.015124+0000","last_clean":"2024-03-18T15:33:43.015124+0000","last_became_active":"2024-03-18T15:33:41.986209+0000","last_became_peered":"2024-03-18T15:33:41.986209+0000","last_unstale":"2024-03-18T15:33:43.015124+0000","last_undegraded":"2024-03-18T15:33:43.015124+0000","last_fullsized":"2024-03-18T15:33:43.015124+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:41:28.732114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.010873+0000","last_change":"2024-03-18T15:33:42.998067+0000","last_active":"2024-03-18T15:33:43.010873+0000","last_peered":"2024-03-18T15:33:43.010873+0000","last_clean":"2024-03-18T15:33:43.010873+0000","last_became_active":"2024-03-18T15:33:42.997755+0000","last_became_peered":"2024-03-18T15:33:42.997755+0000","last_unstale":"2024-03-18T15:33:43.010873+0000","last_undegraded":"2024-03-18T15:33:43.010873+0000","last_fullsized":"2024-03-18T15:33:43.010873+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:06:40.477304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.022995+0000","last_change":"2024-03-18T15:33:41.987009+0000","last_active":"2024-03-18T15:33:43.022995+0000","last_peered":"2024-03-18T15:33:43.022995+0000","last_clean":"2024-03-18T15:33:43.022995+0000","last_became_active":"2024-03-18T15:33:41.986376+0000","last_became_peered":"2024-03-18T15:33:41.986376+0000","last_unstale":"2024-03-18T15:33:43.022995+0000","last_undegraded":"2024-03-18T15:33:43.022995+0000","last_fullsized":"2024-03-18T15:33:43.022995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:26:02.875362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.014781+0000","last_change":"2024-03-18T15:33:43.007612+0000","last_active":"2024-03-18T15:33:43.014781+0000","last_peered":"2024-03-18T15:33:43.014781+0000","last_clean":"2024-03-18T15:33:43.014781+0000","last_became_active":"2024-03-18T15:33:43.006950+0000","last_became_peered":"2024-03-18T15:33:43.006950+0000","last_unstale":"2024-03-18T15:33:43.014781+0000","last_undegraded":"2024-03-18T15:33:43.014781+0000","last_fullsized":"2024-03-18T15:33:43.014781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:11:30.652938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.011210+0000","last_change":"2024-03-18T15:33:41.990256+0000","last_active":"2024-03-18T15:33:43.011210+0000","last_peered":"2024-03-18T15:33:43.011210+0000","last_clean":"2024-03-18T15:33:43.011210+0000","last_became_active":"2024-03-18T15:33:41.989715+0000","last_became_peered":"2024-03-18T15:33:41.989715+0000","last_unstale":"2024-03-18T15:33:43.011210+0000","last_undegraded":"2024-03-18T15:33:43.011210+0000","last_fullsized":"2024-03-18T15:33:43.011210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:23:20.085212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.025666+0000","last_change":"2024-03-18T15:33:43.022519+0000","last_active":"2024-03-18T15:33:43.025666+0000","last_peered":"2024-03-18T15:33:43.025666+0000","last_clean":"2024-03-18T15:33:43.025666+0000","last_became_active":"2024-03-18T15:33:43.022081+0000","last_became_peered":"2024-03-18T15:33:43.022081+0000","last_unstale":"2024-03-18T15:33:43.025666+0000","last_undegraded":"2024-03-18T15:33:43.025666+0000","last_fullsized":"2024-03-18T15:33:43.025666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:21:20.566649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:42.995557+0000","last_change":"2024-03-18T15:33:42.001343+0000","last_active":"2024-03-18T15:33:42.995557+0000","last_peered":"2024-03-18T15:33:42.995557+0000","last_clean":"2024-03-18T15:33:42.995557+0000","last_became_active":"2024-03-18T15:33:42.000948+0000","last_became_peered":"2024-03-18T15:33:42.000948+0000","last_unstale":"2024-03-18T15:33:42.995557+0000","last_undegraded":"2024-03-18T15:33:42.995557+0000","last_fullsized":"2024-03-18T15:33:42.995557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:47:53.438385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.022710+0000","last_change":"2024-03-18T15:33:43.015405+0000","last_active":"2024-03-18T15:33:43.022710+0000","last_peered":"2024-03-18T15:33:43.022710+0000","last_clean":"2024-03-18T15:33:43.022710+0000","last_became_active":"2024-03-18T15:33:43.015094+0000","last_became_peered":"2024-03-18T15:33:43.015094+0000","last_unstale":"2024-03-18T15:33:43.022710+0000","last_undegraded":"2024-03-18T15:33:43.022710+0000","last_fullsized":"2024-03-18T15:33:43.022710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:07:37.987066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":168,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:42.972906+0000","last_change":"2024-03-18T15:33:30.766315+0000","last_active":"2024-03-18T15:33:42.972906+0000","last_peered":"2024-03-18T15:33:42.972906+0000","last_clean":"2024-03-18T15:33:42.972906+0000","last_became_active":"2024-03-18T15:33:30.763939+0000","last_became_peered":"2024-03-18T15:33:30.763939+0000","last_unstale":"2024-03-18T15:33:42.972906+0000","last_undegraded":"2024-03-18T15:33:42.972906+0000","last_fullsized":"2024-03-18T15:33:42.972906+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:29.229911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:29.229911+0000","last_clean_scrub_stamp":"2024-03-18T15:33:29.229911+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:10:21.120371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:42.981172+0000","last_change":"2024-03-18T15:33:41.998201+0000","last_active":"2024-03-18T15:33:42.981172+0000","last_peered":"2024-03-18T15:33:42.981172+0000","last_clean":"2024-03-18T15:33:42.981172+0000","last_became_active":"2024-03-18T15:33:41.997079+0000","last_became_peered":"2024-03-18T15:33:41.997079+0000","last_unstale":"2024-03-18T15:33:42.981172+0000","last_undegraded":"2024-03-18T15:33:42.981172+0000","last_fullsized":"2024-03-18T15:33:42.981172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:38:45.057586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.022917+0000","last_change":"2024-03-18T15:33:43.019262+0000","last_active":"2024-03-18T15:33:43.022917+0000","last_peered":"2024-03-18T15:33:43.022917+0000","last_clean":"2024-03-18T15:33:43.022917+0000","last_became_active":"2024-03-18T15:33:43.018918+0000","last_became_peered":"2024-03-18T15:33:43.018918+0000","last_unstale":"2024-03-18T15:33:43.022917+0000","last_undegraded":"2024-03-18T15:33:43.022917+0000","last_fullsized":"2024-03-18T15:33:43.022917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:43:43.250689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:42.973219+0000","last_change":"2024-03-18T15:33:42.003947+0000","last_active":"2024-03-18T15:33:42.973219+0000","last_peered":"2024-03-18T15:33:42.973219+0000","last_clean":"2024-03-18T15:33:42.973219+0000","last_became_active":"2024-03-18T15:33:42.002250+0000","last_became_peered":"2024-03-18T15:33:42.002250+0000","last_unstale":"2024-03-18T15:33:42.973219+0000","last_undegraded":"2024-03-18T15:33:42.973219+0000","last_fullsized":"2024-03-18T15:33:42.973219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:07:04.089081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.016410+0000","last_change":"2024-03-18T15:33:43.015094+0000","last_active":"2024-03-18T15:33:43.016410+0000","last_peered":"2024-03-18T15:33:43.016410+0000","last_clean":"2024-03-18T15:33:43.016410+0000","last_became_active":"2024-03-18T15:33:43.014596+0000","last_became_peered":"2024-03-18T15:33:43.014596+0000","last_unstale":"2024-03-18T15:33:43.016410+0000","last_undegraded":"2024-03-18T15:33:43.016410+0000","last_fullsized":"2024-03-18T15:33:43.016410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:26:03.887314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:42.994962+0000","last_change":"2024-03-18T15:33:41.991149+0000","last_active":"2024-03-18T15:33:42.994962+0000","last_peered":"2024-03-18T15:33:42.994962+0000","last_clean":"2024-03-18T15:33:42.994962+0000","last_became_active":"2024-03-18T15:33:41.990059+0000","last_became_peered":"2024-03-18T15:33:41.990059+0000","last_unstale":"2024-03-18T15:33:42.994962+0000","last_undegraded":"2024-03-18T15:33:42.994962+0000","last_fullsized":"2024-03-18T15:33:42.994962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:09:14.869832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.006633+0000","last_change":"2024-03-18T15:33:43.000746+0000","last_active":"2024-03-18T15:33:43.006633+0000","last_peered":"2024-03-18T15:33:43.006633+0000","last_clean":"2024-03-18T15:33:43.006633+0000","last_became_active":"2024-03-18T15:33:42.999935+0000","last_became_peered":"2024-03-18T15:33:42.999935+0000","last_unstale":"2024-03-18T15:33:43.006633+0000","last_undegraded":"2024-03-18T15:33:43.006633+0000","last_fullsized":"2024-03-18T15:33:43.006633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:06:40.477304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:42.998144+0000","last_change":"2024-03-18T15:33:42.002409+0000","last_active":"2024-03-18T15:33:42.998144+0000","last_peered":"2024-03-18T15:33:42.998144+0000","last_clean":"2024-03-18T15:33:42.998144+0000","last_became_active":"2024-03-18T15:33:42.000616+0000","last_became_peered":"2024-03-18T15:33:42.000616+0000","last_unstale":"2024-03-18T15:33:42.998144+0000","last_undegraded":"2024-03-18T15:33:42.998144+0000","last_fullsized":"2024-03-18T15:33:42.998144+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:47:53.438385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.010119+0000","last_change":"2024-03-18T15:33:43.005420+0000","last_active":"2024-03-18T15:33:43.010119+0000","last_peered":"2024-03-18T15:33:43.010119+0000","last_clean":"2024-03-18T15:33:43.010119+0000","last_became_active":"2024-03-18T15:33:43.004488+0000","last_became_peered":"2024-03-18T15:33:43.004488+0000","last_unstale":"2024-03-18T15:33:43.010119+0000","last_undegraded":"2024-03-18T15:33:43.010119+0000","last_fullsized":"2024-03-18T15:33:43.010119+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:46:34.352988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:42.970787+0000","last_change":"2024-03-18T15:33:41.990472+0000","last_active":"2024-03-18T15:33:42.970787+0000","last_peered":"2024-03-18T15:33:42.970787+0000","last_clean":"2024-03-18T15:33:42.970787+0000","last_became_active":"2024-03-18T15:33:41.989782+0000","last_became_peered":"2024-03-18T15:33:41.989782+0000","last_unstale":"2024-03-18T15:33:42.970787+0000","last_undegraded":"2024-03-18T15:33:42.970787+0000","last_fullsized":"2024-03-18T15:33:42.970787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:35:45.651780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.018367+0000","last_change":"2024-03-18T15:33:43.013011+0000","last_active":"2024-03-18T15:33:43.018367+0000","last_peered":"2024-03-18T15:33:43.018367+0000","last_clean":"2024-03-18T15:33:43.018367+0000","last_became_active":"2024-03-18T15:33:43.012334+0000","last_became_peered":"2024-03-18T15:33:43.012334+0000","last_unstale":"2024-03-18T15:33:43.018367+0000","last_undegraded":"2024-03-18T15:33:43.018367+0000","last_fullsized":"2024-03-18T15:33:43.018367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:13:38.420228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:42.979759+0000","last_change":"2024-03-18T15:33:42.011960+0000","last_active":"2024-03-18T15:33:42.979759+0000","last_peered":"2024-03-18T15:33:42.979759+0000","last_clean":"2024-03-18T15:33:42.979759+0000","last_became_active":"2024-03-18T15:33:42.010760+0000","last_became_peered":"2024-03-18T15:33:42.010760+0000","last_unstale":"2024-03-18T15:33:42.979759+0000","last_undegraded":"2024-03-18T15:33:42.979759+0000","last_fullsized":"2024-03-18T15:33:42.979759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:29:33.713515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:42.992570+0000","last_change":"2024-03-18T15:33:42.985474+0000","last_active":"2024-03-18T15:33:42.992570+0000","last_peered":"2024-03-18T15:33:42.992570+0000","last_clean":"2024-03-18T15:33:42.992570+0000","last_became_active":"2024-03-18T15:33:42.984659+0000","last_became_peered":"2024-03-18T15:33:42.984659+0000","last_unstale":"2024-03-18T15:33:42.992570+0000","last_undegraded":"2024-03-18T15:33:42.992570+0000","last_fullsized":"2024-03-18T15:33:42.992570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:29:34.725467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:42.971572+0000","last_change":"2024-03-18T15:33:41.994297+0000","last_active":"2024-03-18T15:33:42.971572+0000","last_peered":"2024-03-18T15:33:42.971572+0000","last_clean":"2024-03-18T15:33:42.971572+0000","last_became_active":"2024-03-18T15:33:41.993906+0000","last_became_peered":"2024-03-18T15:33:41.993906+0000","last_unstale":"2024-03-18T15:33:42.971572+0000","last_undegraded":"2024-03-18T15:33:42.971572+0000","last_fullsized":"2024-03-18T15:33:42.971572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:07:04.089081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.022744+0000","last_change":"2024-03-18T15:33:43.018039+0000","last_active":"2024-03-18T15:33:43.022744+0000","last_peered":"2024-03-18T15:33:43.022744+0000","last_clean":"2024-03-18T15:33:43.022744+0000","last_became_active":"2024-03-18T15:33:43.017680+0000","last_became_peered":"2024-03-18T15:33:43.017680+0000","last_unstale":"2024-03-18T15:33:43.022744+0000","last_undegraded":"2024-03-18T15:33:43.022744+0000","last_fullsized":"2024-03-18T15:33:43.022744+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T15:35:23.735580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.002591+0000","last_change":"2024-03-18T15:33:42.009920+0000","last_active":"2024-03-18T15:33:44.002591+0000","last_peered":"2024-03-18T15:33:44.002591+0000","last_clean":"2024-03-18T15:33:44.002591+0000","last_became_active":"2024-03-18T15:33:42.009394+0000","last_became_peered":"2024-03-18T15:33:42.009394+0000","last_unstale":"2024-03-18T15:33:44.002591+0000","last_undegraded":"2024-03-18T15:33:44.002591+0000","last_fullsized":"2024-03-18T15:33:44.002591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:27:05.573887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.004940+0000","last_change":"2024-03-18T15:33:42.999878+0000","last_active":"2024-03-18T15:33:43.004940+0000","last_peered":"2024-03-18T15:33:43.004940+0000","last_clean":"2024-03-18T15:33:43.004940+0000","last_became_active":"2024-03-18T15:33:42.999140+0000","last_became_peered":"2024-03-18T15:33:42.999140+0000","last_unstale":"2024-03-18T15:33:43.004940+0000","last_undegraded":"2024-03-18T15:33:43.004940+0000","last_fullsized":"2024-03-18T15:33:43.004940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:38:46.069538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:42.972728+0000","last_change":"2024-03-18T15:33:42.009110+0000","last_active":"2024-03-18T15:33:42.972728+0000","last_peered":"2024-03-18T15:33:42.972728+0000","last_clean":"2024-03-18T15:33:42.972728+0000","last_became_active":"2024-03-18T15:33:42.008110+0000","last_became_peered":"2024-03-18T15:33:42.008110+0000","last_unstale":"2024-03-18T15:33:42.972728+0000","last_undegraded":"2024-03-18T15:33:42.972728+0000","last_fullsized":"2024-03-18T15:33:42.972728+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:23:20.085212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.011113+0000","last_change":"2024-03-18T15:33:43.006171+0000","last_active":"2024-03-18T15:33:43.011113+0000","last_peered":"2024-03-18T15:33:43.011113+0000","last_clean":"2024-03-18T15:33:43.011113+0000","last_became_active":"2024-03-18T15:33:43.005851+0000","last_became_peered":"2024-03-18T15:33:43.005851+0000","last_unstale":"2024-03-18T15:33:43.011113+0000","last_undegraded":"2024-03-18T15:33:43.011113+0000","last_fullsized":"2024-03-18T15:33:43.011113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:44:50.201644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:42.971290+0000","last_change":"2024-03-18T15:33:41.997380+0000","last_active":"2024-03-18T15:33:42.971290+0000","last_peered":"2024-03-18T15:33:42.971290+0000","last_clean":"2024-03-18T15:33:42.971290+0000","last_became_active":"2024-03-18T15:33:41.997101+0000","last_became_peered":"2024-03-18T15:33:41.997101+0000","last_unstale":"2024-03-18T15:33:42.971290+0000","last_undegraded":"2024-03-18T15:33:42.971290+0000","last_fullsized":"2024-03-18T15:33:42.971290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:33:11.480578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.022578+0000","last_change":"2024-03-18T15:33:43.012575+0000","last_active":"2024-03-18T15:33:43.022578+0000","last_peered":"2024-03-18T15:33:43.022578+0000","last_clean":"2024-03-18T15:33:43.022578+0000","last_became_active":"2024-03-18T15:33:42.996972+0000","last_became_peered":"2024-03-18T15:33:42.996972+0000","last_unstale":"2024-03-18T15:33:43.022578+0000","last_undegraded":"2024-03-18T15:33:43.022578+0000","last_fullsized":"2024-03-18T15:33:43.022578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:33:12.492530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:42.971759+0000","last_change":"2024-03-18T15:33:41.990440+0000","last_active":"2024-03-18T15:33:42.971759+0000","last_peered":"2024-03-18T15:33:42.971759+0000","last_clean":"2024-03-18T15:33:42.971759+0000","last_became_active":"2024-03-18T15:33:41.989808+0000","last_became_peered":"2024-03-18T15:33:41.989808+0000","last_unstale":"2024-03-18T15:33:42.971759+0000","last_undegraded":"2024-03-18T15:33:42.971759+0000","last_fullsized":"2024-03-18T15:33:42.971759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:14:42.231258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.022384+0000","last_change":"2024-03-18T15:33:43.012652+0000","last_active":"2024-03-18T15:33:43.022384+0000","last_peered":"2024-03-18T15:33:43.022384+0000","last_clean":"2024-03-18T15:33:43.022384+0000","last_became_active":"2024-03-18T15:33:43.010391+0000","last_became_peered":"2024-03-18T15:33:43.010391+0000","last_unstale":"2024-03-18T15:33:43.022384+0000","last_undegraded":"2024-03-18T15:33:43.022384+0000","last_fullsized":"2024-03-18T15:33:43.022384+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:07:05.101032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.011549+0000","last_change":"2024-03-18T15:33:42.013549+0000","last_active":"2024-03-18T15:33:43.011549+0000","last_peered":"2024-03-18T15:33:43.011549+0000","last_clean":"2024-03-18T15:33:43.011549+0000","last_became_active":"2024-03-18T15:33:42.012707+0000","last_became_peered":"2024-03-18T15:33:42.012707+0000","last_unstale":"2024-03-18T15:33:43.011549+0000","last_undegraded":"2024-03-18T15:33:43.011549+0000","last_fullsized":"2024-03-18T15:33:43.011549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:14:42.231258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.002049+0000","last_change":"2024-03-18T15:33:43.003144+0000","last_active":"2024-03-18T15:33:44.002049+0000","last_peered":"2024-03-18T15:33:44.002049+0000","last_clean":"2024-03-18T15:33:44.002049+0000","last_became_active":"2024-03-18T15:33:43.002771+0000","last_became_peered":"2024-03-18T15:33:43.002771+0000","last_unstale":"2024-03-18T15:33:44.002049+0000","last_undegraded":"2024-03-18T15:33:44.002049+0000","last_fullsized":"2024-03-18T15:33:44.002049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:50:43.021257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.018726+0000","last_change":"2024-03-18T15:33:43.014403+0000","last_active":"2024-03-18T15:33:43.018726+0000","last_peered":"2024-03-18T15:33:43.018726+0000","last_clean":"2024-03-18T15:33:43.018726+0000","last_became_active":"2024-03-18T15:33:43.012667+0000","last_became_peered":"2024-03-18T15:33:43.012667+0000","last_unstale":"2024-03-18T15:33:43.018726+0000","last_undegraded":"2024-03-18T15:33:43.018726+0000","last_fullsized":"2024-03-18T15:33:43.018726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:43:00.095618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.019002+0000","last_change":"2024-03-18T15:33:42.009430+0000","last_active":"2024-03-18T15:33:43.019002+0000","last_peered":"2024-03-18T15:33:43.019002+0000","last_clean":"2024-03-18T15:33:43.019002+0000","last_became_active":"2024-03-18T15:33:42.008331+0000","last_became_peered":"2024-03-18T15:33:42.008331+0000","last_unstale":"2024-03-18T15:33:43.019002+0000","last_undegraded":"2024-03-18T15:33:43.019002+0000","last_fullsized":"2024-03-18T15:33:43.019002+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:33:11.480578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.020201+0000","last_change":"2024-03-18T15:33:43.017670+0000","last_active":"2024-03-18T15:33:43.020201+0000","last_peered":"2024-03-18T15:33:43.020201+0000","last_clean":"2024-03-18T15:33:43.020201+0000","last_became_active":"2024-03-18T15:33:43.017297+0000","last_became_peered":"2024-03-18T15:33:43.017297+0000","last_unstale":"2024-03-18T15:33:43.020201+0000","last_undegraded":"2024-03-18T15:33:43.020201+0000","last_fullsized":"2024-03-18T15:33:43.020201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:57:58.192115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.011284+0000","last_change":"2024-03-18T15:33:42.009132+0000","last_active":"2024-03-18T15:33:43.011284+0000","last_peered":"2024-03-18T15:33:43.011284+0000","last_clean":"2024-03-18T15:33:43.011284+0000","last_became_active":"2024-03-18T15:33:42.008343+0000","last_became_peered":"2024-03-18T15:33:42.008343+0000","last_unstale":"2024-03-18T15:33:43.011284+0000","last_undegraded":"2024-03-18T15:33:43.011284+0000","last_fullsized":"2024-03-18T15:33:43.011284+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T15:35:22.723629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.021872+0000","last_change":"2024-03-18T15:33:43.021277+0000","last_active":"2024-03-18T15:33:43.021872+0000","last_peered":"2024-03-18T15:33:43.021872+0000","last_clean":"2024-03-18T15:33:43.021872+0000","last_became_active":"2024-03-18T15:33:43.020857+0000","last_became_peered":"2024-03-18T15:33:43.020857+0000","last_unstale":"2024-03-18T15:33:43.021872+0000","last_undegraded":"2024-03-18T15:33:43.021872+0000","last_fullsized":"2024-03-18T15:33:43.021872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:43:00.095618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.022483+0000","last_change":"2024-03-18T15:33:41.992048+0000","last_active":"2024-03-18T15:33:43.022483+0000","last_peered":"2024-03-18T15:33:43.022483+0000","last_clean":"2024-03-18T15:33:43.022483+0000","last_became_active":"2024-03-18T15:33:41.989055+0000","last_became_peered":"2024-03-18T15:33:41.989055+0000","last_unstale":"2024-03-18T15:33:43.022483+0000","last_undegraded":"2024-03-18T15:33:43.022483+0000","last_fullsized":"2024-03-18T15:33:43.022483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:46:33.341037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.019817+0000","last_change":"2024-03-18T15:33:42.996569+0000","last_active":"2024-03-18T15:33:43.019817+0000","last_peered":"2024-03-18T15:33:43.019817+0000","last_clean":"2024-03-18T15:33:43.019817+0000","last_became_active":"2024-03-18T15:33:42.996234+0000","last_became_peered":"2024-03-18T15:33:42.996234+0000","last_unstale":"2024-03-18T15:33:43.019817+0000","last_undegraded":"2024-03-18T15:33:43.019817+0000","last_fullsized":"2024-03-18T15:33:43.019817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:44:50.201644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.019674+0000","last_change":"2024-03-18T15:33:42.010405+0000","last_active":"2024-03-18T15:33:43.019674+0000","last_peered":"2024-03-18T15:33:43.019674+0000","last_clean":"2024-03-18T15:33:43.019674+0000","last_became_active":"2024-03-18T15:33:42.009883+0000","last_became_peered":"2024-03-18T15:33:42.009883+0000","last_unstale":"2024-03-18T15:33:43.019674+0000","last_undegraded":"2024-03-18T15:33:43.019674+0000","last_fullsized":"2024-03-18T15:33:43.019674+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:10:06.802810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":75067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":200704,"data_stored":67245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":794624,"data_stored":658105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":184320,"data_stored":60380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35548,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336292,"statfs":{"total":96636764160,"available":96600363008,"internally_reserved":0,"allocated":208896,"data_stored":68767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35524,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336316,"statfs":{"total":96636764160,"available":96600387584,"internally_reserved":0,"allocated":184320,"data_stored":60380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":204800,"data_stored":67315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36124,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335716,"statfs":{"total":96636764160,"available":96599773184,"internally_reserved":0,"allocated":802816,"data_stored":658161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-18T15:33:47.392 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-18T15:33:47.392 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-18T15:33:47.392 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-03-18T15:33:47.392 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-03-18T15:33:47.393 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-03-18T15:33:47.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.545+0000 7f775c249640 1 Processor -- start 2024-03-18T15:33:47.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.545+0000 7f775c249640 1 -- start start 2024-03-18T15:33:47.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.549+0000 7f775c249640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f775413f260 0x7f775413f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:47.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.549+0000 7f775c249640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f775414dc70 0x7f7754150060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:47.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.549+0000 7f775c249640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77540b2ab0 con 0x7f775406d2d0 2024-03-18T15:33:47.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.549+0000 7f775c249640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f77540728c0 con 0x7f775414dc70 2024-03-18T15:33:47.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.549+0000 7f775c249640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f77540b0ea0 con 0x7f775413f260 2024-03-18T15:33:47.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.549+0000 7f775a7e6640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f775414dc70 0x7f7754150060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:47.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.549+0000 7f775a7e6640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f775414dc70 0x7f7754150060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38356/0 (socket says 172.21.15.5:38356) 2024-03-18T15:33:47.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.549+0000 7f775a7e6640 1 -- 172.21.15.5:0/551255861 learned_addr learned my addr 172.21.15.5:0/551255861 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:47.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.549+0000 7f77597e4640 1 --2- 172.21.15.5:0/551255861 >> v2:172.21.15.80:3300/0 conn(0x7f775413f260 0x7f775413f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:47.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.549+0000 7f775a7e6640 1 -- 172.21.15.5:0/551255861 >> v2:172.21.15.80:3300/0 conn(0x7f775413f260 msgr2=0x7f775413f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:47.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.549+0000 7f775a7e6640 1 --2- 172.21.15.5:0/551255861 >> v2:172.21.15.80:3300/0 conn(0x7f775413f260 0x7f775413f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:47.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.549+0000 7f775a7e6640 1 -- 172.21.15.5:0/551255861 >> v1:172.21.15.80:6789/0 conn(0x7f775406d2d0 legacy=0x7f77540b23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:47.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.549+0000 7f775a7e6640 1 -- 172.21.15.5:0/551255861 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f775406d6b0 con 0x7f775414dc70 2024-03-18T15:33:47.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.549+0000 7f775a7e6640 1 --2- 172.21.15.5:0/551255861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f775414dc70 0x7f7754150060 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f775409eaa0 tx=0x7f774c01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e867deb6551b29a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:47.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.549+0000 7f7758fe3640 1 -- 172.21.15.5:0/551255861 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c0a5710 con 0x7f775414dc70 2024-03-18T15:33:47.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.549+0000 7f7758fe3640 1 -- 172.21.15.5:0/551255861 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f774c0a58b0 con 0x7f775414dc70 2024-03-18T15:33:47.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.549+0000 7f7758fe3640 1 -- 172.21.15.5:0/551255861 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c0a5c80 con 0x7f775414dc70 2024-03-18T15:33:47.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.549+0000 7f775c249640 1 -- 172.21.15.5:0/551255861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f775414dc70 msgr2=0x7f7754150060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:47.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.549+0000 7f775c249640 1 --2- 172.21.15.5:0/551255861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f775414dc70 0x7f7754150060 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f775409eaa0 tx=0x7f774c01c960 comp rx=0 tx=0).stop 2024-03-18T15:33:47.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.553+0000 7f775c249640 1 -- 172.21.15.5:0/551255861 shutdown_connections 2024-03-18T15:33:47.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.553+0000 7f775c249640 1 --2- 172.21.15.5:0/551255861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f775414dc70 0x7f7754150060 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:47.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.553+0000 7f775c249640 1 --2- 172.21.15.5:0/551255861 >> v2:172.21.15.80:3300/0 conn(0x7f775413f260 0x7f775413f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:47.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.553+0000 7f775c249640 1 -- 172.21.15.5:0/551255861 >> 172.21.15.5:0/551255861 conn(0x7f77540b5670 msgr2=0x7f77540b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:47.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.553+0000 7f775c249640 1 -- 172.21.15.5:0/551255861 shutdown_connections 2024-03-18T15:33:47.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.553+0000 7f775c249640 1 -- 172.21.15.5:0/551255861 wait complete. 2024-03-18T15:33:47.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.553+0000 7f775c249640 1 Processor -- start 2024-03-18T15:33:47.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.553+0000 7f775c249640 1 -- start start 2024-03-18T15:33:47.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.553+0000 7f775c249640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f775406d2d0 0x7f77540ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:47.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.553+0000 7f775c249640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f775413f260 0x7f775412fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:47.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.553+0000 7f775c249640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f775414dc70 0x7f77540af3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:47.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.553+0000 7f775c249640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f77540b0960 con 0x7f775414dc70 2024-03-18T15:33:47.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.553+0000 7f775c249640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f77540b07e0 con 0x7f775413f260 2024-03-18T15:33:47.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.553+0000 7f775c249640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f77540b5140 con 0x7f775406d2d0 2024-03-18T15:33:47.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.553+0000 7f775a7e6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f775414dc70 0x7f77540af3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:47.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.553+0000 7f775a7e6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f775414dc70 0x7f77540af3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38360/0 (socket says 172.21.15.5:38360) 2024-03-18T15:33:47.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.553+0000 7f775a7e6640 1 -- 172.21.15.5:0/1947235587 learned_addr learned my addr 172.21.15.5:0/1947235587 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:47.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.553+0000 7f7759fe5640 1 --2- 172.21.15.5:0/1947235587 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f775406d2d0 0x7f77540ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:47.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.553+0000 7f77597e4640 1 --2- 172.21.15.5:0/1947235587 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f775413f260 0x7f775412fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:47.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.553+0000 7f775a7e6640 1 -- 172.21.15.5:0/1947235587 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f775406d2d0 msgr2=0x7f77540ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:47.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.553+0000 7f775a7e6640 1 --2- 172.21.15.5:0/1947235587 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f775406d2d0 0x7f77540ae680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:47.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.553+0000 7f775a7e6640 1 -- 172.21.15.5:0/1947235587 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f775413f260 msgr2=0x7f775412fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:47.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.553+0000 7f775a7e6640 1 --2- 172.21.15.5:0/1947235587 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f775413f260 0x7f775412fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:47.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.553+0000 7f775a7e6640 1 -- 172.21.15.5:0/1947235587 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77540adbe0 con 0x7f775414dc70 2024-03-18T15:33:47.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.553+0000 7f77597e4640 1 --2- 172.21.15.5:0/1947235587 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f775413f260 0x7f775412fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:33:47.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.553+0000 7f775a7e6640 1 --2- 172.21.15.5:0/1947235587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f775414dc70 0x7f77540af3c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f774c019570 tx=0x7f774c019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:47.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.553+0000 7f77467fc640 1 -- 172.21.15.5:0/1947235587 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c0bc020 con 0x7f775414dc70 2024-03-18T15:33:47.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.553+0000 7f77467fc640 1 -- 172.21.15.5:0/1947235587 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f774c0b8670 con 0x7f775414dc70 2024-03-18T15:33:47.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.553+0000 7f775c249640 1 -- 172.21.15.5:0/1947235587 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f775414fe40 con 0x7f775414dc70 2024-03-18T15:33:47.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.553+0000 7f77467fc640 1 -- 172.21.15.5:0/1947235587 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c009510 con 0x7f775414dc70 2024-03-18T15:33:47.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.553+0000 7f775c249640 1 -- 172.21.15.5:0/1947235587 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77540ae160 con 0x7f775414dc70 2024-03-18T15:33:47.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.553+0000 7f775c249640 1 -- 172.21.15.5:0/1947235587 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f775404f550 con 0x7f775414dc70 2024-03-18T15:33:47.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.553+0000 7f77467fc640 1 -- 172.21.15.5:0/1947235587 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f774c0bf020 con 0x7f775414dc70 2024-03-18T15:33:47.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.557+0000 7f77467fc640 1 --2- 172.21.15.5:0/1947235587 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f7738067660 0x7f7738069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:47.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.557+0000 7f77467fc640 1 -- 172.21.15.5:0/1947235587 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f774c120820 con 0x7f775414dc70 2024-03-18T15:33:47.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.557+0000 7f7759fe5640 1 --2- 172.21.15.5:0/1947235587 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f7738067660 0x7f7738069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:47.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.557+0000 7f7759fe5640 1 --2- 172.21.15.5:0/1947235587 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f7738067660 0x7f7738069b20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7748000f70 tx=0x7f7748009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:47.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.561+0000 7f77467fc640 1 -- 172.21.15.5:0/1947235587 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f774c0c6030 con 0x7f775414dc70 2024-03-18T15:33:47.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.753+0000 7f775c249640 1 -- 172.21.15.5:0/1947235587 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f7754142ff0 con 0x7f775414dc70 2024-03-18T15:33:47.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.757+0000 7f77467fc640 1 -- 172.21.15.5:0/1947235587 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+166519 (secure 0 0 0) 0x7f774c0165e0 con 0x7f775414dc70 2024-03-18T15:33:47.762 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:33:47.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.761+0000 7f775c249640 1 -- 172.21.15.5:0/1947235587 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f7738067660 msgr2=0x7f7738069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:47.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.761+0000 7f775c249640 1 --2- 172.21.15.5:0/1947235587 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f7738067660 0x7f7738069b20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7748000f70 tx=0x7f7748009040 comp rx=0 tx=0).stop 2024-03-18T15:33:47.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.761+0000 7f775c249640 1 -- 172.21.15.5:0/1947235587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f775414dc70 msgr2=0x7f77540af3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:47.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.761+0000 7f775c249640 1 --2- 172.21.15.5:0/1947235587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f775414dc70 0x7f77540af3c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f774c019570 tx=0x7f774c019700 comp rx=0 tx=0).stop 2024-03-18T15:33:47.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.761+0000 7f775c249640 1 -- 172.21.15.5:0/1947235587 shutdown_connections 2024-03-18T15:33:47.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.761+0000 7f775c249640 1 --2- 172.21.15.5:0/1947235587 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f7738067660 0x7f7738069b20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:47.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.761+0000 7f775c249640 1 --2- 172.21.15.5:0/1947235587 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f775414dc70 0x7f77540af3c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:47.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.761+0000 7f775c249640 1 --2- 172.21.15.5:0/1947235587 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f775413f260 0x7f775412fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:47.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.761+0000 7f775c249640 1 --2- 172.21.15.5:0/1947235587 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f775406d2d0 0x7f77540ae680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:47.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.761+0000 7f775c249640 1 -- 172.21.15.5:0/1947235587 >> 172.21.15.5:0/1947235587 conn(0x7f77540b5670 msgr2=0x7f775413c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:47.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.761+0000 7f775c249640 1 -- 172.21.15.5:0/1947235587 shutdown_connections 2024-03-18T15:33:47.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.761+0000 7f775c249640 1 -- 172.21.15.5:0/1947235587 wait complete. 2024-03-18T15:33:47.778 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":5,"flags":0,"active_gid":4110,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3362034860},{"type":"v1","addr":"172.21.15.5:6833","nonce":3362034860}]},"active_addr":"172.21.15.5:6833/3362034860","active_change":"2024-03-18T15:33:22.875354+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4102,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":1859984758}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":1759993522}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":2386453334}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":744665053}]}]} 2024-03-18T15:33:47.781 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-03-18T15:33:47.781 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-03-18T15:33:47.781 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:33:47.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.897+0000 7fbc84e68640 1 Processor -- start 2024-03-18T15:33:47.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.897+0000 7fbc84e68640 1 -- start start 2024-03-18T15:33:47.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.901+0000 7fbc84e68640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fbc80072b40 0x7fbc8006d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:47.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.901+0000 7fbc84e68640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc800afb70 con 0x7fbc800b26a0 2024-03-18T15:33:47.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.901+0000 7fbc84e68640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc800b4a20 con 0x7fbc800af790 2024-03-18T15:33:47.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.901+0000 7fbc84e68640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fbc8006d5b0 con 0x7fbc80072b40 2024-03-18T15:33:47.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.901+0000 7fbc7ed9d640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fbc800b26a0 0x7fbc800b2af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:57218/0 (socket says 172.21.15.5:57218) 2024-03-18T15:33:47.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.901+0000 7fbc7ed9d640 1 -- 172.21.15.5:0/1275800896 learned_addr learned my addr 172.21.15.5:0/1275800896 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:33:47.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.901+0000 7fbc7dd9b640 1 --2- 172.21.15.5:0/1275800896 >> v2:172.21.15.80:3300/0 conn(0x7fbc80072b40 0x7fbc8006d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:47.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.901+0000 7fbc7d59a640 1 -- 172.21.15.5:0/1275800896 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 692347929 0 0) 0x7fbc800afb70 con 0x7fbc800b26a0 2024-03-18T15:33:47.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.901+0000 7fbc7d59a640 1 -- 172.21.15.5:0/1275800896 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc6c003530 con 0x7fbc800b26a0 2024-03-18T15:33:47.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.901+0000 7fbc7d59a640 1 -- 172.21.15.5:0/1275800896 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3344385885 0 0) 0x7fbc6c003530 con 0x7fbc800b26a0 2024-03-18T15:33:47.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.901+0000 7fbc7d59a640 1 -- 172.21.15.5:0/1275800896 >> v2:172.21.15.80:3300/0 conn(0x7fbc80072b40 msgr2=0x7fbc8006d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:47.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.901+0000 7fbc7d59a640 1 --2- 172.21.15.5:0/1275800896 >> v2:172.21.15.80:3300/0 conn(0x7fbc80072b40 0x7fbc8006d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:47.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.901+0000 7fbc7d59a640 1 -- 172.21.15.5:0/1275800896 >> v1:172.21.15.80:6789/0 conn(0x7fbc800af790 legacy=0x7fbc80147780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:47.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.901+0000 7fbc7d59a640 1 -- 172.21.15.5:0/1275800896 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc800b4d00 con 0x7fbc800b26a0 2024-03-18T15:33:47.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.901+0000 7fbc7d59a640 1 -- 172.21.15.5:0/1275800896 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3837549003 0 0) 0x7fbc78001a00 con 0x7fbc800b26a0 2024-03-18T15:33:47.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.901+0000 7fbc7d59a640 1 -- 172.21.15.5:0/1275800896 >> v1:172.21.15.5:6789/0 conn(0x7fbc800b26a0 legacy=0x7fbc800b2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:47.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.901+0000 7fbc7d59a640 1 --2- 172.21.15.5:0/1275800896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc6c003aa0 0x7fbc6c003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:47.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.901+0000 7fbc7d59a640 1 --2- 172.21.15.5:0/1275800896 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc6c004810 0x7fbc6c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:47.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.901+0000 7fbc7d59a640 1 --2- 172.21.15.5:0/1275800896 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbc6c0052a0 0x7fbc6c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:47.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.901+0000 7fbc7d59a640 1 -- 172.21.15.5:0/1275800896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8006d5b0 con 0x7fbc6c003aa0 2024-03-18T15:33:47.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.901+0000 7fbc7d59a640 1 -- 172.21.15.5:0/1275800896 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fbc6c00c1f0 con 0x7fbc6c004810 2024-03-18T15:33:47.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.901+0000 7fbc7d59a640 1 -- 172.21.15.5:0/1275800896 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fbc6c00c3f0 con 0x7fbc6c0052a0 2024-03-18T15:33:47.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.901+0000 7fbc7e59c640 1 --2- 172.21.15.5:0/1275800896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc6c003aa0 0x7fbc6c003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:47.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.901+0000 7fbc7dd9b640 1 --2- 172.21.15.5:0/1275800896 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc6c004810 0x7fbc6c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:47.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.901+0000 7fbc7ed9d640 1 --2- 172.21.15.5:0/1275800896 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbc6c0052a0 0x7fbc6c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:47.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.901+0000 7fbc7e59c640 1 -- 172.21.15.5:0/1275800896 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbc6c0052a0 msgr2=0x7fbc6c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:47.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.901+0000 7fbc7e59c640 1 --2- 172.21.15.5:0/1275800896 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbc6c0052a0 0x7fbc6c00b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:47.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.901+0000 7fbc7e59c640 1 -- 172.21.15.5:0/1275800896 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc6c004810 msgr2=0x7fbc6c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:47.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.901+0000 7fbc7e59c640 1 --2- 172.21.15.5:0/1275800896 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc6c004810 0x7fbc6c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:47.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.901+0000 7fbc7e59c640 1 -- 172.21.15.5:0/1275800896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc6c003530 con 0x7fbc6c003aa0 2024-03-18T15:33:47.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.901+0000 7fbc7ed9d640 1 -- 172.21.15.5:0/1275800896 reap_dead start 2024-03-18T15:33:47.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.901+0000 7fbc7e59c640 1 --2- 172.21.15.5:0/1275800896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc6c003aa0 0x7fbc6c003ec0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbc68001780 tx=0x7fbc6801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e91f1c5c4e9ff4f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:47.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.901+0000 7fbc7d59a640 1 -- 172.21.15.5:0/1275800896 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc680013a0 con 0x7fbc6c003aa0 2024-03-18T15:33:47.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.901+0000 7fbc7d59a640 1 -- 172.21.15.5:0/1275800896 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc680052f0 con 0x7fbc6c003aa0 2024-03-18T15:33:47.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.901+0000 7fbc7d59a640 1 -- 172.21.15.5:0/1275800896 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc68003840 con 0x7fbc6c003aa0 2024-03-18T15:33:47.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.905+0000 7fbc84e68640 1 -- 172.21.15.5:0/1275800896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc6c003aa0 msgr2=0x7fbc6c003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:47.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.905+0000 7fbc84e68640 1 --2- 172.21.15.5:0/1275800896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc6c003aa0 0x7fbc6c003ec0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbc68001780 tx=0x7fbc6801c400 comp rx=0 tx=0).stop 2024-03-18T15:33:47.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.905+0000 7fbc84e68640 1 -- 172.21.15.5:0/1275800896 shutdown_connections 2024-03-18T15:33:47.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.905+0000 7fbc84e68640 1 --2- 172.21.15.5:0/1275800896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc6c003aa0 0x7fbc6c003ec0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:47.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.905+0000 7fbc84e68640 1 -- 172.21.15.5:0/1275800896 >> 172.21.15.5:0/1275800896 conn(0x7fbc800b5670 msgr2=0x7fbc800b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:47.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.905+0000 7fbc84e68640 1 -- 172.21.15.5:0/1275800896 shutdown_connections 2024-03-18T15:33:47.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.905+0000 7fbc84e68640 1 -- 172.21.15.5:0/1275800896 wait complete. 2024-03-18T15:33:47.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.905+0000 7fbc84e68640 1 Processor -- start 2024-03-18T15:33:47.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.905+0000 7fbc84e68640 1 -- start start 2024-03-18T15:33:47.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.905+0000 7fbc84e68640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc6c003aa0 0x7fbc801d3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:47.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.905+0000 7fbc84e68640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc800b6970 0x7fbc801e06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:47.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.905+0000 7fbc84e68640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbc8013d0a0 0x7fbc8013d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:47.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.905+0000 7fbc84e68640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbc80147990 con 0x7fbc800b6970 2024-03-18T15:33:47.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.905+0000 7fbc84e68640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fbc80072f20 con 0x7fbc6c003aa0 2024-03-18T15:33:47.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.905+0000 7fbc84e68640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fbc80147b10 con 0x7fbc8013d0a0 2024-03-18T15:33:47.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.905+0000 7fbc7dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc800b6970 0x7fbc801e06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:47.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.905+0000 7fbc7dd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc800b6970 0x7fbc801e06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38370/0 (socket says 172.21.15.5:38370) 2024-03-18T15:33:47.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.905+0000 7fbc7dd9b640 1 -- 172.21.15.5:0/1599257382 learned_addr learned my addr 172.21.15.5:0/1599257382 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:47.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.905+0000 7fbc7e59c640 1 --2- 172.21.15.5:0/1599257382 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc6c003aa0 0x7fbc801d3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:47.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.905+0000 7fbc7ed9d640 1 --2- 172.21.15.5:0/1599257382 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbc8013d0a0 0x7fbc8013d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:47.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.905+0000 7fbc7dd9b640 1 -- 172.21.15.5:0/1599257382 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbc8013d0a0 msgr2=0x7fbc8013d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:47.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.905+0000 7fbc7dd9b640 1 --2- 172.21.15.5:0/1599257382 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbc8013d0a0 0x7fbc8013d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:47.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.905+0000 7fbc7dd9b640 1 -- 172.21.15.5:0/1599257382 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc6c003aa0 msgr2=0x7fbc801d3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:47.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.905+0000 7fbc7dd9b640 1 --2- 172.21.15.5:0/1599257382 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc6c003aa0 0x7fbc801d3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:47.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.905+0000 7fbc7dd9b640 1 -- 172.21.15.5:0/1599257382 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc800b6e30 con 0x7fbc800b6970 2024-03-18T15:33:47.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.905+0000 7fbc7dd9b640 1 --2- 172.21.15.5:0/1599257382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc800b6970 0x7fbc801e06a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbc700116c0 tx=0x7fbc70011b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:47.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.905+0000 7fbc76ffd640 1 -- 172.21.15.5:0/1599257382 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7000f960 con 0x7fbc800b6970 2024-03-18T15:33:47.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.905+0000 7fbc76ffd640 1 -- 172.21.15.5:0/1599257382 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc7000fb00 con 0x7fbc800b6970 2024-03-18T15:33:47.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.905+0000 7fbc84e68640 1 -- 172.21.15.5:0/1599257382 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc800b7110 con 0x7fbc800b6970 2024-03-18T15:33:47.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.905+0000 7fbc76ffd640 1 -- 172.21.15.5:0/1599257382 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc70015460 con 0x7fbc800b6970 2024-03-18T15:33:47.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.905+0000 7fbc84e68640 1 -- 172.21.15.5:0/1599257382 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc80141e30 con 0x7fbc800b6970 2024-03-18T15:33:47.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.905+0000 7fbc84e68640 1 -- 172.21.15.5:0/1599257382 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc8004f550 con 0x7fbc800b6970 2024-03-18T15:33:47.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.909+0000 7fbc76ffd640 1 -- 172.21.15.5:0/1599257382 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fbc70015600 con 0x7fbc800b6970 2024-03-18T15:33:47.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.909+0000 7fbc76ffd640 1 --2- 172.21.15.5:0/1599257382 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fbc60067690 0x7fbc60069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:47.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.909+0000 7fbc76ffd640 1 -- 172.21.15.5:0/1599257382 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fbc70086080 con 0x7fbc800b6970 2024-03-18T15:33:47.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.909+0000 7fbc7e59c640 1 --2- 172.21.15.5:0/1599257382 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fbc60067690 0x7fbc60069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:47.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.909+0000 7fbc7e59c640 1 --2- 172.21.15.5:0/1599257382 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fbc60067690 0x7fbc60069b50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbc68001750 tx=0x7fbc680011a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:47.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:47.913+0000 7fbc76ffd640 1 -- 172.21.15.5:0/1599257382 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fbc70019030 con 0x7fbc800b6970 2024-03-18T15:33:48.011 INFO:tasks.ceph.mds.b.smithi080.stderr:2024-03-18T15:33:48.008+0000 7fddb8a20640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-18T15:33:48.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.073+0000 7fbc84e68640 1 -- 172.21.15.5:0/1599257382 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbc800afb70 con 0x7fbc800b6970 2024-03-18T15:33:48.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.073+0000 7fbc76ffd640 1 -- 172.21.15.5:0/1599257382 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16449 (secure 0 0 0) 0x7fbc70056d90 con 0x7fbc800b6970 2024-03-18T15:33:48.077 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:33:48.077 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":21,"fsid":"ef54fe03-5aa7-4eb2-9b1e-66c9cc345b94","created":"2024-03-18T15:33:18.086614+0000","modified":"2024-03-18T15:33:43.984050+0000","last_up_change":"2024-03-18T15:33:29.229911+0000","last_in_change":"2024-03-18T15:33:20.323840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:28.963186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T15:33:40.533646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T15:33:41.323519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df1c754b-af59-4d16-8919-fbb6386b3c48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6804","nonce":976264754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6808","nonce":976264754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6815","nonce":976264754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6811","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6813","nonce":976264754}]},"public_addr":"172.21.15.5:6804/976264754","cluster_addr":"172.21.15.5:6808/976264754","heartbeat_back_addr":"172.21.15.5:6815/976264754","heartbeat_front_addr":"172.21.15.5:6813/976264754","state":["exists","up"]},{"osd":1,"uuid":"b9a08b5e-8af4-4144-bdba-d69ed1ac627a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6801","nonce":3901761126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6803","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6805","nonce":3901761126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6812","nonce":3901761126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6807","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6809","nonce":3901761126}]},"public_addr":"172.21.15.5:6801/3901761126","cluster_addr":"172.21.15.5:6805/3901761126","heartbeat_back_addr":"172.21.15.5:6812/3901761126","heartbeat_front_addr":"172.21.15.5:6809/3901761126","state":["exists","up"]},{"osd":2,"uuid":"c3e8bb76-e382-439d-b380-62ef92f1255a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6817","nonce":502968448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6819","nonce":502968448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6823","nonce":502968448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6821","nonce":502968448}]},"public_addr":"172.21.15.5:6817/502968448","cluster_addr":"172.21.15.5:6819/502968448","heartbeat_back_addr":"172.21.15.5:6823/502968448","heartbeat_front_addr":"172.21.15.5:6821/502968448","state":["exists","up"]},{"osd":3,"uuid":"e8c94625-f2cd-4eba-b53d-2e7356e1c02a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6825","nonce":2425932735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6827","nonce":2425932735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6831","nonce":2425932735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6829","nonce":2425932735}]},"public_addr":"172.21.15.5:6825/2425932735","cluster_addr":"172.21.15.5:6827/2425932735","heartbeat_back_addr":"172.21.15.5:6831/2425932735","heartbeat_front_addr":"172.21.15.5:6829/2425932735","state":["exists","up"]},{"osd":4,"uuid":"0099dd52-6265-4d32-82e4-3ef16e73add0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6817","nonce":976514285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6819","nonce":976514285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6823","nonce":976514285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6821","nonce":976514285}]},"public_addr":"172.21.15.80:6817/976514285","cluster_addr":"172.21.15.80:6819/976514285","heartbeat_back_addr":"172.21.15.80:6823/976514285","heartbeat_front_addr":"172.21.15.80:6821/976514285","state":["exists","up"]},{"osd":5,"uuid":"ed10c84f-8c39-41cc-9b64-7c482a94a48e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6825","nonce":627153447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6827","nonce":627153447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6831","nonce":627153447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6829","nonce":627153447}]},"public_addr":"172.21.15.80:6825/627153447","cluster_addr":"172.21.15.80:6827/627153447","heartbeat_back_addr":"172.21.15.80:6831/627153447","heartbeat_front_addr":"172.21.15.80:6829/627153447","state":["exists","up"]},{"osd":6,"uuid":"c7a3538a-7048-4fba-8781-b440f03c25db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6803","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6805","nonce":488342527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6807","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6809","nonce":488342527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6815","nonce":488342527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6811","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6813","nonce":488342527}]},"public_addr":"172.21.15.80:6805/488342527","cluster_addr":"172.21.15.80:6809/488342527","heartbeat_back_addr":"172.21.15.80:6815/488342527","heartbeat_front_addr":"172.21.15.80:6813/488342527","state":["exists","up"]},{"osd":7,"uuid":"f9418104-3a97-4110-bc3a-1f8151d0f757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6801","nonce":982831666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6804","nonce":982831666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6812","nonce":982831666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6808","nonce":982831666}]},"public_addr":"172.21.15.80:6801/982831666","cluster_addr":"172.21.15.80:6804/982831666","heartbeat_back_addr":"172.21.15.80:6812/982831666","heartbeat_front_addr":"172.21.15.80:6808/982831666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.970828+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.858116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.987556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.864603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.756293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.591502+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:33:48.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.077+0000 7fbc84e68640 1 -- 172.21.15.5:0/1599257382 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fbc60067690 msgr2=0x7fbc60069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.077+0000 7fbc84e68640 1 --2- 172.21.15.5:0/1599257382 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fbc60067690 0x7fbc60069b50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbc68001750 tx=0x7fbc680011a0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.077+0000 7fbc84e68640 1 -- 172.21.15.5:0/1599257382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc800b6970 msgr2=0x7fbc801e06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.077+0000 7fbc84e68640 1 --2- 172.21.15.5:0/1599257382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc800b6970 0x7fbc801e06a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbc700116c0 tx=0x7fbc70011b80 comp rx=0 tx=0).stop 2024-03-18T15:33:48.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.077+0000 7fbc84e68640 1 -- 172.21.15.5:0/1599257382 shutdown_connections 2024-03-18T15:33:48.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.077+0000 7fbc84e68640 1 --2- 172.21.15.5:0/1599257382 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fbc60067690 0x7fbc60069b50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.077+0000 7fbc84e68640 1 --2- 172.21.15.5:0/1599257382 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbc8013d0a0 0x7fbc8013d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.077+0000 7fbc84e68640 1 --2- 172.21.15.5:0/1599257382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc800b6970 0x7fbc801e06a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.077+0000 7fbc84e68640 1 --2- 172.21.15.5:0/1599257382 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc6c003aa0 0x7fbc801d3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.077+0000 7fbc84e68640 1 -- 172.21.15.5:0/1599257382 >> 172.21.15.5:0/1599257382 conn(0x7fbc800b5670 msgr2=0x7fbc800690d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:48.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.077+0000 7fbc84e68640 1 -- 172.21.15.5:0/1599257382 shutdown_connections 2024-03-18T15:33:48.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.077+0000 7fbc84e68640 1 -- 172.21.15.5:0/1599257382 wait complete. 2024-03-18T15:33:48.096 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-03-18T15:33:48.097 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:33:48.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.217+0000 7f817a806640 1 Processor -- start 2024-03-18T15:33:48.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.217+0000 7f817a806640 1 -- start start 2024-03-18T15:33:48.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.221+0000 7f817a806640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f817406d2d0 0x7f81740b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.221+0000 7f817a806640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81740b0ea0 con 0x7f817413f6a0 2024-03-18T15:33:48.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.221+0000 7f817a806640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f817406d6b0 con 0x7f817413f260 2024-03-18T15:33:48.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.221+0000 7f817a806640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f81740728c0 con 0x7f817406d2d0 2024-03-18T15:33:48.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.221+0000 7f8173fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f817406d2d0 0x7f81740b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.221+0000 7f8178da3640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f817413f6a0 0x7f8174145ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:57224/0 (socket says 172.21.15.5:57224) 2024-03-18T15:33:48.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.221+0000 7f8178da3640 1 -- 172.21.15.5:0/2582413746 learned_addr learned my addr 172.21.15.5:0/2582413746 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:33:48.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.221+0000 7f8173fff640 1 -- 172.21.15.5:0/2582413746 >> v1:172.21.15.80:6789/0 conn(0x7f817413f260 legacy=0x7f81741453a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:48.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.221+0000 7f8173fff640 1 -- 172.21.15.5:0/2582413746 >> v1:172.21.15.5:6789/0 conn(0x7f817413f6a0 legacy=0x7f8174145ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.221+0000 7f8173fff640 1 -- 172.21.15.5:0/2582413746 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f817413faf0 con 0x7f817406d2d0 2024-03-18T15:33:48.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.221+0000 7f8172ffd640 1 -- 172.21.15.5:0/2582413746 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1013383718 0 0) 0x7f81740b0ea0 con 0x7f817413f6a0 2024-03-18T15:33:48.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.221+0000 7f8173fff640 1 --2- 172.21.15.5:0/2582413746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f817406d2d0 0x7f81740b23a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f81640097c0 tx=0x7f816401c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e43042ab281482a1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.221+0000 7f8172ffd640 1 -- 172.21.15.5:0/2582413746 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81640b1070 con 0x7f817406d2d0 2024-03-18T15:33:48.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.221+0000 7f8172ffd640 1 -- 172.21.15.5:0/2582413746 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81640ac600 con 0x7f817406d2d0 2024-03-18T15:33:48.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.221+0000 7f8172ffd640 1 -- 172.21.15.5:0/2582413746 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81640ac9d0 con 0x7f817406d2d0 2024-03-18T15:33:48.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.221+0000 7f817a806640 1 -- 172.21.15.5:0/2582413746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f817406d2d0 msgr2=0x7f81740b23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.221+0000 7f817a806640 1 --2- 172.21.15.5:0/2582413746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f817406d2d0 0x7f81740b23a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f81640097c0 tx=0x7f816401c3a0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.221+0000 7f817a806640 1 -- 172.21.15.5:0/2582413746 shutdown_connections 2024-03-18T15:33:48.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.221+0000 7f817a806640 1 --2- 172.21.15.5:0/2582413746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f817406d2d0 0x7f81740b23a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.221+0000 7f817a806640 1 -- 172.21.15.5:0/2582413746 >> 172.21.15.5:0/2582413746 conn(0x7f81740b5670 msgr2=0x7f81740b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:48.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.225+0000 7f817a806640 1 -- 172.21.15.5:0/2582413746 shutdown_connections 2024-03-18T15:33:48.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.225+0000 7f817a806640 1 -- 172.21.15.5:0/2582413746 wait complete. 2024-03-18T15:33:48.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.225+0000 7f817a806640 1 Processor -- start 2024-03-18T15:33:48.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.225+0000 7f817a806640 1 -- start start 2024-03-18T15:33:48.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.225+0000 7f817a806640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f817406d2d0 0x7f8174071ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.225+0000 7f817a806640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f817413f260 0x7f81741e3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.225+0000 7f817a806640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f817413f6a0 0x7f817413b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.225+0000 7f817a806640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f81740b01f0 con 0x7f817406d2d0 2024-03-18T15:33:48.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.225+0000 7f817a806640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f81740b0070 con 0x7f817413f260 2024-03-18T15:33:48.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.225+0000 7f817a806640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f81740b07e0 con 0x7f817413f6a0 2024-03-18T15:33:48.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.225+0000 7f8173fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f817406d2d0 0x7f8174071ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.225+0000 7f8173fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f817406d2d0 0x7f8174071ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38388/0 (socket says 172.21.15.5:38388) 2024-03-18T15:33:48.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.225+0000 7f8173fff640 1 -- 172.21.15.5:0/2787381054 learned_addr learned my addr 172.21.15.5:0/2787381054 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:48.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.225+0000 7f81737fe640 1 --2- 172.21.15.5:0/2787381054 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f817413f260 0x7f81741e3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.225+0000 7f8178da3640 1 --2- 172.21.15.5:0/2787381054 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f817413f6a0 0x7f817413b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.225+0000 7f8173fff640 1 -- 172.21.15.5:0/2787381054 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f817413f6a0 msgr2=0x7f817413b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.225+0000 7f8173fff640 1 --2- 172.21.15.5:0/2787381054 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f817413f6a0 0x7f817413b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.225+0000 7f8173fff640 1 -- 172.21.15.5:0/2787381054 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f817413f260 msgr2=0x7f81741e3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.225+0000 7f8173fff640 1 --2- 172.21.15.5:0/2787381054 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f817413f260 0x7f81741e3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.225+0000 7f8173fff640 1 -- 172.21.15.5:0/2787381054 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81740712b0 con 0x7f817406d2d0 2024-03-18T15:33:48.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.225+0000 7f8178da3640 1 --2- 172.21.15.5:0/2787381054 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f817413f6a0 0x7f817413b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:33:48.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.225+0000 7f8173fff640 1 --2- 172.21.15.5:0/2787381054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f817406d2d0 0x7f8174071ca0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f81640097c0 tx=0x7f816401abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.225+0000 7f8170ff9640 1 -- 172.21.15.5:0/2787381054 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81640bb020 con 0x7f817406d2d0 2024-03-18T15:33:48.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.225+0000 7f8170ff9640 1 -- 172.21.15.5:0/2787381054 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81640b7640 con 0x7f817406d2d0 2024-03-18T15:33:48.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.225+0000 7f817a806640 1 -- 172.21.15.5:0/2787381054 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81741e2e10 con 0x7f817406d2d0 2024-03-18T15:33:48.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.225+0000 7f8170ff9640 1 -- 172.21.15.5:0/2787381054 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81640b1040 con 0x7f817406d2d0 2024-03-18T15:33:48.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.225+0000 7f817a806640 1 -- 172.21.15.5:0/2787381054 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81741e3b50 con 0x7f817406d2d0 2024-03-18T15:33:48.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.225+0000 7f817a806640 1 -- 172.21.15.5:0/2787381054 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f817404f550 con 0x7f817406d2d0 2024-03-18T15:33:48.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.225+0000 7f8170ff9640 1 -- 172.21.15.5:0/2787381054 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f81640be020 con 0x7f817406d2d0 2024-03-18T15:33:48.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.229+0000 7f8170ff9640 1 --2- 172.21.15.5:0/2787381054 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f8160067690 0x7f8160069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.229+0000 7f8170ff9640 1 -- 172.21.15.5:0/2787381054 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f816411fc40 con 0x7f817406d2d0 2024-03-18T15:33:48.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.229+0000 7f81737fe640 1 --2- 172.21.15.5:0/2787381054 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f8160067690 0x7f8160069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.229+0000 7f81737fe640 1 --2- 172.21.15.5:0/2787381054 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f8160067690 0x7f8160069b50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8174149a10 tx=0x7f8168009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.233+0000 7f8170ff9640 1 -- 172.21.15.5:0/2787381054 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f81640c5030 con 0x7f817406d2d0 2024-03-18T15:33:48.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.385+0000 7f817a806640 1 -- 172.21.15.5:0/2787381054 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f817413bd70 con 0x7f817406d2d0 2024-03-18T15:33:48.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.389+0000 7f8170ff9640 1 -- 172.21.15.5:0/2787381054 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16449 (secure 0 0 0) 0x7f8164016050 con 0x7f817406d2d0 2024-03-18T15:33:48.390 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:33:48.391 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":21,"fsid":"ef54fe03-5aa7-4eb2-9b1e-66c9cc345b94","created":"2024-03-18T15:33:18.086614+0000","modified":"2024-03-18T15:33:43.984050+0000","last_up_change":"2024-03-18T15:33:29.229911+0000","last_in_change":"2024-03-18T15:33:20.323840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:28.963186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T15:33:40.533646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T15:33:41.323519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df1c754b-af59-4d16-8919-fbb6386b3c48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6804","nonce":976264754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6808","nonce":976264754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6815","nonce":976264754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6811","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6813","nonce":976264754}]},"public_addr":"172.21.15.5:6804/976264754","cluster_addr":"172.21.15.5:6808/976264754","heartbeat_back_addr":"172.21.15.5:6815/976264754","heartbeat_front_addr":"172.21.15.5:6813/976264754","state":["exists","up"]},{"osd":1,"uuid":"b9a08b5e-8af4-4144-bdba-d69ed1ac627a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6801","nonce":3901761126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6803","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6805","nonce":3901761126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6812","nonce":3901761126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6807","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6809","nonce":3901761126}]},"public_addr":"172.21.15.5:6801/3901761126","cluster_addr":"172.21.15.5:6805/3901761126","heartbeat_back_addr":"172.21.15.5:6812/3901761126","heartbeat_front_addr":"172.21.15.5:6809/3901761126","state":["exists","up"]},{"osd":2,"uuid":"c3e8bb76-e382-439d-b380-62ef92f1255a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6817","nonce":502968448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6819","nonce":502968448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6823","nonce":502968448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6821","nonce":502968448}]},"public_addr":"172.21.15.5:6817/502968448","cluster_addr":"172.21.15.5:6819/502968448","heartbeat_back_addr":"172.21.15.5:6823/502968448","heartbeat_front_addr":"172.21.15.5:6821/502968448","state":["exists","up"]},{"osd":3,"uuid":"e8c94625-f2cd-4eba-b53d-2e7356e1c02a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6825","nonce":2425932735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6827","nonce":2425932735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6831","nonce":2425932735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6829","nonce":2425932735}]},"public_addr":"172.21.15.5:6825/2425932735","cluster_addr":"172.21.15.5:6827/2425932735","heartbeat_back_addr":"172.21.15.5:6831/2425932735","heartbeat_front_addr":"172.21.15.5:6829/2425932735","state":["exists","up"]},{"osd":4,"uuid":"0099dd52-6265-4d32-82e4-3ef16e73add0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6817","nonce":976514285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6819","nonce":976514285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6823","nonce":976514285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6821","nonce":976514285}]},"public_addr":"172.21.15.80:6817/976514285","cluster_addr":"172.21.15.80:6819/976514285","heartbeat_back_addr":"172.21.15.80:6823/976514285","heartbeat_front_addr":"172.21.15.80:6821/976514285","state":["exists","up"]},{"osd":5,"uuid":"ed10c84f-8c39-41cc-9b64-7c482a94a48e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6825","nonce":627153447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6827","nonce":627153447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6831","nonce":627153447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6829","nonce":627153447}]},"public_addr":"172.21.15.80:6825/627153447","cluster_addr":"172.21.15.80:6827/627153447","heartbeat_back_addr":"172.21.15.80:6831/627153447","heartbeat_front_addr":"172.21.15.80:6829/627153447","state":["exists","up"]},{"osd":6,"uuid":"c7a3538a-7048-4fba-8781-b440f03c25db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6803","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6805","nonce":488342527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6807","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6809","nonce":488342527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6815","nonce":488342527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6811","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6813","nonce":488342527}]},"public_addr":"172.21.15.80:6805/488342527","cluster_addr":"172.21.15.80:6809/488342527","heartbeat_back_addr":"172.21.15.80:6815/488342527","heartbeat_front_addr":"172.21.15.80:6813/488342527","state":["exists","up"]},{"osd":7,"uuid":"f9418104-3a97-4110-bc3a-1f8151d0f757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6801","nonce":982831666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6804","nonce":982831666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6812","nonce":982831666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6808","nonce":982831666}]},"public_addr":"172.21.15.80:6801/982831666","cluster_addr":"172.21.15.80:6804/982831666","heartbeat_back_addr":"172.21.15.80:6812/982831666","heartbeat_front_addr":"172.21.15.80:6808/982831666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.970828+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.858116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.987556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.864603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.756293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.591502+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:33:48.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.389+0000 7f817a806640 1 -- 172.21.15.5:0/2787381054 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f8160067690 msgr2=0x7f8160069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.389+0000 7f817a806640 1 --2- 172.21.15.5:0/2787381054 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f8160067690 0x7f8160069b50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8174149a10 tx=0x7f8168009040 comp rx=0 tx=0).stop 2024-03-18T15:33:48.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.389+0000 7f817a806640 1 -- 172.21.15.5:0/2787381054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f817406d2d0 msgr2=0x7f8174071ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.389+0000 7f817a806640 1 --2- 172.21.15.5:0/2787381054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f817406d2d0 0x7f8174071ca0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f81640097c0 tx=0x7f816401abe0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.389+0000 7f817a806640 1 -- 172.21.15.5:0/2787381054 shutdown_connections 2024-03-18T15:33:48.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.389+0000 7f817a806640 1 --2- 172.21.15.5:0/2787381054 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f8160067690 0x7f8160069b50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.389+0000 7f817a806640 1 --2- 172.21.15.5:0/2787381054 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f817413f6a0 0x7f817413b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.389+0000 7f817a806640 1 --2- 172.21.15.5:0/2787381054 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f817413f260 0x7f81741e3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.389+0000 7f817a806640 1 --2- 172.21.15.5:0/2787381054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f817406d2d0 0x7f8174071ca0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.389+0000 7f817a806640 1 -- 172.21.15.5:0/2787381054 >> 172.21.15.5:0/2787381054 conn(0x7f81740b5670 msgr2=0x7f817406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:48.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.389+0000 7f817a806640 1 -- 172.21.15.5:0/2787381054 shutdown_connections 2024-03-18T15:33:48.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.389+0000 7f817a806640 1 -- 172.21.15.5:0/2787381054 wait complete. 2024-03-18T15:33:48.408 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-03-18T15:33:48.408 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-03-18T15:33:48.409 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-03-18T15:33:48.409 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-03-18T15:33:48.410 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-03-18T15:33:48.410 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-03-18T15:33:48.410 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-03-18T15:33:48.411 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-03-18T15:33:48.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.573+0000 7f1f6b2bf640 1 Processor -- start 2024-03-18T15:33:48.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.573+0000 7f1f6b2bf640 1 -- start start 2024-03-18T15:33:48.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.577+0000 7f1f6b2bf640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1f640b4ee0 0x7f1f641470e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.577+0000 7f1f6b2bf640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1f640b1540 0x7f1f6414dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.577+0000 7f1f6b2bf640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f640afe60 con 0x7f1f640b1160 2024-03-18T15:33:48.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.577+0000 7f1f6b2bf640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f1f640b0140 con 0x7f1f640b4ee0 2024-03-18T15:33:48.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.577+0000 7f1f6b2bf640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f1f640b0ab0 con 0x7f1f640b1540 2024-03-18T15:33:48.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.577+0000 7f1f6905b640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f1f640b1160 0x7f1f640b5ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.5:44084/0 (socket says 172.21.15.5:44084) 2024-03-18T15:33:48.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.577+0000 7f1f6905b640 1 -- 172.21.15.5:0/554352477 learned_addr learned my addr 172.21.15.5:0/554352477 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:33:48.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.577+0000 7f1f6885a640 1 --2- 172.21.15.5:0/554352477 >> v2:172.21.15.5:3300/0 conn(0x7f1f640b4ee0 0x7f1f641470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.577+0000 7f1f6885a640 1 -- 172.21.15.5:0/554352477 >> v2:172.21.15.80:3300/0 conn(0x7f1f640b1540 msgr2=0x7f1f6414dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:48.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.577+0000 7f1f6985c640 1 --2- 172.21.15.5:0/554352477 >> v2:172.21.15.80:3300/0 conn(0x7f1f640b1540 0x7f1f6414dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.577+0000 7f1f6885a640 1 --2- 172.21.15.5:0/554352477 >> v2:172.21.15.80:3300/0 conn(0x7f1f640b1540 0x7f1f6414dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.577+0000 7f1f6885a640 1 -- 172.21.15.5:0/554352477 >> v1:172.21.15.80:6789/0 conn(0x7f1f640b1160 legacy=0x7f1f640b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.577+0000 7f1f6885a640 1 -- 172.21.15.5:0/554352477 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f640b0d90 con 0x7f1f640b4ee0 2024-03-18T15:33:48.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.581+0000 7f1f6985c640 1 --2- 172.21.15.5:0/554352477 >> v2:172.21.15.80:3300/0 conn(0x7f1f640b1540 0x7f1f6414dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:33:48.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.581+0000 7f1f6885a640 1 --2- 172.21.15.5:0/554352477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f640b4ee0 0x7f1f641470e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1f640b47d0 tx=0x7f1f6001bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9930b9fa37435a97 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.581+0000 7f1f5bfff640 1 -- 172.21.15.5:0/554352477 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f600ab070 con 0x7f1f640b4ee0 2024-03-18T15:33:48.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.581+0000 7f1f5bfff640 1 -- 172.21.15.5:0/554352477 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f64147830 con 0x7f1f640b4ee0 2024-03-18T15:33:48.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.581+0000 7f1f5bfff640 1 -- 172.21.15.5:0/554352477 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f600a5840 con 0x7f1f640b4ee0 2024-03-18T15:33:48.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.581+0000 7f1f6b2bf640 1 -- 172.21.15.5:0/554352477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f640b4ee0 msgr2=0x7f1f641470e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.581+0000 7f1f6b2bf640 1 --2- 172.21.15.5:0/554352477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f640b4ee0 0x7f1f641470e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1f640b47d0 tx=0x7f1f6001bf80 comp rx=0 tx=0).stop 2024-03-18T15:33:48.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.581+0000 7f1f6b2bf640 1 -- 172.21.15.5:0/554352477 shutdown_connections 2024-03-18T15:33:48.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.581+0000 7f1f6b2bf640 1 --2- 172.21.15.5:0/554352477 >> v2:172.21.15.80:3300/0 conn(0x7f1f640b1540 0x7f1f6414dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.581+0000 7f1f6b2bf640 1 --2- 172.21.15.5:0/554352477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f640b4ee0 0x7f1f641470e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.581+0000 7f1f6b2bf640 1 -- 172.21.15.5:0/554352477 >> 172.21.15.5:0/554352477 conn(0x7f1f64066fc0 msgr2=0x7f1f640673d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:48.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.581+0000 7f1f6b2bf640 1 -- 172.21.15.5:0/554352477 shutdown_connections 2024-03-18T15:33:48.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.581+0000 7f1f6b2bf640 1 -- 172.21.15.5:0/554352477 wait complete. 2024-03-18T15:33:48.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.585+0000 7f1f6b2bf640 1 Processor -- start 2024-03-18T15:33:48.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.585+0000 7f1f6b2bf640 1 -- start start 2024-03-18T15:33:48.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.585+0000 7f1f6b2bf640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1f640b1160 0x7f1f64070a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.585+0000 7f1f6b2bf640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f640b1540 0x7f1f641e24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.585+0000 7f1f6b2bf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f640b4ee0 0x7f1f6413e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.585+0000 7f1f6b2bf640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1f64152640 con 0x7f1f640b4ee0 2024-03-18T15:33:48.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.585+0000 7f1f6b2bf640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1f641524c0 con 0x7f1f640b1540 2024-03-18T15:33:48.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.585+0000 7f1f6b2bf640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f1f641527c0 con 0x7f1f640b1160 2024-03-18T15:33:48.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.585+0000 7f1f6885a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f640b1540 0x7f1f641e24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.585+0000 7f1f6905b640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1f640b1160 0x7f1f64070a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.585+0000 7f1f6885a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f640b1540 0x7f1f641e24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:58936/0 (socket says 172.21.15.5:58936) 2024-03-18T15:33:48.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.585+0000 7f1f6885a640 1 -- 172.21.15.5:0/1994964951 learned_addr learned my addr 172.21.15.5:0/1994964951 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:48.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.585+0000 7f1f6985c640 1 --2- 172.21.15.5:0/1994964951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f640b4ee0 0x7f1f6413e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.585+0000 7f1f6885a640 1 -- 172.21.15.5:0/1994964951 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1f640b1160 msgr2=0x7f1f64070a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.585+0000 7f1f6885a640 1 --2- 172.21.15.5:0/1994964951 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1f640b1160 0x7f1f64070a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.585+0000 7f1f6885a640 1 -- 172.21.15.5:0/1994964951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f640b4ee0 msgr2=0x7f1f6413e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.585+0000 7f1f6885a640 1 --2- 172.21.15.5:0/1994964951 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f640b4ee0 0x7f1f6413e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.585+0000 7f1f6885a640 1 -- 172.21.15.5:0/1994964951 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f64148450 con 0x7f1f640b1540 2024-03-18T15:33:48.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.589+0000 7f1f6885a640 1 --2- 172.21.15.5:0/1994964951 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f640b1540 0x7f1f641e24d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1f60019dc0 tx=0x7f1f60019040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.589+0000 7f1f59ffb640 1 -- 172.21.15.5:0/1994964951 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f600ab070 con 0x7f1f640b1540 2024-03-18T15:33:48.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.589+0000 7f1f6b2bf640 1 -- 172.21.15.5:0/1994964951 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f64140c10 con 0x7f1f640b1540 2024-03-18T15:33:48.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.589+0000 7f1f6b2bf640 1 -- 172.21.15.5:0/1994964951 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f64140930 con 0x7f1f640b1540 2024-03-18T15:33:48.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.589+0000 7f1f59ffb640 1 -- 172.21.15.5:0/1994964951 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f60003820 con 0x7f1f640b1540 2024-03-18T15:33:48.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.589+0000 7f1f59ffb640 1 -- 172.21.15.5:0/1994964951 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f60003bf0 con 0x7f1f640b1540 2024-03-18T15:33:48.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.589+0000 7f1f6b2bf640 1 -- 172.21.15.5:0/1994964951 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1f44001040 con 0x7f1f640b1540 2024-03-18T15:33:48.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.589+0000 7f1f59ffb640 1 -- 172.21.15.5:0/1994964951 <== mon.1 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f1f600ae070 con 0x7f1f640b1540 2024-03-18T15:33:48.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.589+0000 7f54561c9640 1 Processor -- start 2024-03-18T15:33:48.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.589+0000 7f54561c9640 1 -- start start 2024-03-18T15:33:48.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.589+0000 7f1f59ffb640 1 --2- 172.21.15.5:0/1994964951 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f1f48067690 0x7f1f48069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.589+0000 7f1f59ffb640 1 -- 172.21.15.5:0/1994964951 <== mon.1 v2:172.21.15.80:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1f6011efb0 con 0x7f1f640b1540 2024-03-18T15:33:48.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7f1f59ffb640 1 --2- 172.21.15.5:0/1994964951 >> [v2:172.21.15.5:6800/3901761126,v1:172.21.15.5:6801/3901761126] conn(0x7f1f4806f910 0x7f1f48071d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7f1f59ffb640 1 -- 172.21.15.5:0/1994964951 --> [v2:172.21.15.5:6800/3901761126,v1:172.21.15.5:6801/3901761126] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1f60007e50 con 0x7f1f4806f910 2024-03-18T15:33:48.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7f1f59ffb640 1 -- 172.21.15.5:0/1994964951 <== mon.1 v2:172.21.15.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f1f6011f3a0 con 0x7f1f640b1540 2024-03-18T15:33:48.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7fa67afe1640 1 Processor -- start 2024-03-18T15:33:48.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7f54561c9640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5450145570 0x7f54500b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7fa67afe1640 1 -- start start 2024-03-18T15:33:48.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7f54561c9640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f545014dbd0 0x7f545014ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7f54561c9640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54500b30f0 con 0x7f54500b5480 2024-03-18T15:33:48.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7f54561c9640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f54500b3940 con 0x7f545014dbd0 2024-03-18T15:33:48.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7f54561c9640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f54500b0eb0 con 0x7f5450145570 2024-03-18T15:33:48.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7f544f7fe640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5450145570 0x7f54500b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7fe55aebf640 1 Processor -- start 2024-03-18T15:33:48.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7f544f7fe640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5450145570 0x7f54500b4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:52812/0 (socket says 172.21.15.5:52812) 2024-03-18T15:33:48.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7f544f7fe640 1 -- 172.21.15.5:0/1547569341 learned_addr learned my addr 172.21.15.5:0/1547569341 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:48.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7f544ffff640 1 --2- 172.21.15.5:0/1547569341 >> v2:172.21.15.5:3300/0 conn(0x7f545014dbd0 0x7f545014ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7fc20f2ce640 1 Processor -- start 2024-03-18T15:33:48.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7fe55aebf640 1 -- start start 2024-03-18T15:33:48.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7f1f6985c640 1 --2- 172.21.15.5:0/1994964951 >> [v2:172.21.15.5:6800/3901761126,v1:172.21.15.5:6801/3901761126] conn(0x7f1f4806f910 0x7f1f48071d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7fc20f2ce640 1 -- start start 2024-03-18T15:33:48.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7f1f6905b640 1 --2- 172.21.15.5:0/1994964951 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f1f48067690 0x7f1f48069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7fc20f2ce640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc2080b4d50 0x7fc208147cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7fc20f2ce640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc20806b8c0 con 0x7fc20806b4e0 2024-03-18T15:33:48.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7fc20f2ce640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2080b51e0 con 0x7fc208072d50 2024-03-18T15:33:48.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7fc20f2ce640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fc208072830 con 0x7fc2080b4d50 2024-03-18T15:33:48.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7fa67afe1640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa6741479a0 0x7fa674147d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7fe55aebf640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe5541479a0 0x7fe554147d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7fe55aebf640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe5541482c0 0x7fe554070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7fa67afe1640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa6741482c0 0x7fa674070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7fe55aebf640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe554071240 0x7fe55413dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7fe558c5b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe5541479a0 0x7fe554147d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7fe558c5b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe5541479a0 0x7fe554147d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38432/0 (socket says 172.21.15.5:38432) 2024-03-18T15:33:48.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7fc20d86b640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fc208072d50 0x7fc208148230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.5:44092/0 (socket says 172.21.15.5:44092) 2024-03-18T15:33:48.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7fc20d86b640 1 -- 172.21.15.5:0/2406007395 learned_addr learned my addr 172.21.15.5:0/2406007395 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:33:48.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7fa67afe1640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6740716d0 con 0x7fa674071240 2024-03-18T15:33:48.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7fa67afe1640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fa6740702b0 con 0x7fa6741479a0 2024-03-18T15:33:48.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7fe558c5b640 1 -- 172.21.15.5:0/2767311152 learned_addr learned my addr 172.21.15.5:0/2767311152 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:48.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7fe553fff640 1 --2- 172.21.15.5:0/2767311152 >> v2:172.21.15.80:3300/0 conn(0x7fe5541482c0 0x7fe554070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7fa67afe1640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fa674070870 con 0x7fa6741482c0 2024-03-18T15:33:48.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7fe55aebf640 1 -- 172.21.15.5:0/2767311152 --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fe5540702b0 con 0x7fe5541479a0 2024-03-18T15:33:48.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7fa67957e640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa674071240 0x7fa67413dfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:57246/0 (socket says 172.21.15.5:57246) 2024-03-18T15:33:48.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7fa673fff640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa6741482c0 0x7fa674070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7fa67957e640 1 -- 172.21.15.5:0/3431074685 learned_addr learned my addr 172.21.15.5:0/3431074685 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:33:48.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7fe55945c640 1 --2- 172.21.15.5:0/2767311152 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe554071240 0x7fe55413dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7fa678d7d640 1 --2- 172.21.15.5:0/3431074685 >> v2:172.21.15.80:3300/0 conn(0x7fa6741479a0 0x7fa674147d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7fe55aebf640 1 -- 172.21.15.5:0/2767311152 --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fe554070870 con 0x7fe5541482c0 2024-03-18T15:33:48.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7f1f6985c640 1 --2- 172.21.15.5:0/1994964951 >> [v2:172.21.15.5:6800/3901761126,v1:172.21.15.5:6801/3901761126] conn(0x7f1f4806f910 0x7f1f48071d30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7fe55aebf640 1 -- 172.21.15.5:0/2767311152 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fe554072790 con 0x7fe554071240 2024-03-18T15:33:48.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7fc20c869640 1 --2- 172.21.15.5:0/2406007395 >> v2:172.21.15.5:3300/0 conn(0x7fc2080b4d50 0x7fc208147cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7fe558c5b640 1 -- 172.21.15.5:0/2767311152 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe554071240 msgr2=0x7fe55413dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:48.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7fe558c5b640 1 --2- 172.21.15.5:0/2767311152 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe554071240 0x7fe55413dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7fe558c5b640 1 -- 172.21.15.5:0/2767311152 >> v2:172.21.15.80:3300/0 conn(0x7fe5541482c0 msgr2=0x7fe554070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:48.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7fe558c5b640 1 --2- 172.21.15.5:0/2767311152 >> v2:172.21.15.80:3300/0 conn(0x7fe5541482c0 0x7fe554070d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7fe558c5b640 1 -- 172.21.15.5:0/2767311152 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5540716d0 con 0x7fe5541479a0 2024-03-18T15:33:48.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7f1f6905b640 1 --2- 172.21.15.5:0/1994964951 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f1f48067690 0x7f1f48069b50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0012f0 tx=0x7f1f5c0017b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.597+0000 7f1f59ffb640 1 -- 172.21.15.5:0/1994964951 <== osd.1 v2:172.21.15.5:6800/3901761126 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1f60007e50 con 0x7f1f4806f910 2024-03-18T15:33:48.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7f544f7fe640 1 -- 172.21.15.5:0/1547569341 >> v2:172.21.15.5:3300/0 conn(0x7f545014dbd0 msgr2=0x7f545014ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:48.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7f544f7fe640 1 --2- 172.21.15.5:0/1547569341 >> v2:172.21.15.5:3300/0 conn(0x7f545014dbd0 0x7f545014ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7f544f7fe640 1 -- 172.21.15.5:0/1547569341 >> v1:172.21.15.5:6789/0 conn(0x7f54500b5480 legacy=0x7f54500b1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7f544f7fe640 1 -- 172.21.15.5:0/1547569341 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5450145950 con 0x7f5450145570 2024-03-18T15:33:48.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7fc1fffff640 1 -- 172.21.15.5:0/2406007395 <== mon.1 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3192137551 0 0) 0x7fc2080b51e0 con 0x7fc208072d50 2024-03-18T15:33:48.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.597+0000 7fc1fffff640 1 -- 172.21.15.5:0/2406007395 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc1f4003530 con 0x7fc208072d50 2024-03-18T15:33:48.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.597+0000 7fc1fffff640 1 -- 172.21.15.5:0/2406007395 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1870238743 0 0) 0x7fc20806b8c0 con 0x7fc20806b4e0 2024-03-18T15:33:48.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.597+0000 7fc20c869640 1 -- 172.21.15.5:0/2406007395 >> v1:172.21.15.80:6789/0 conn(0x7fc208072d50 legacy=0x7fc208148230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.597+0000 7fc20c869640 1 -- 172.21.15.5:0/2406007395 >> v1:172.21.15.5:6789/0 conn(0x7fc20806b4e0 legacy=0x7fc20809f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.597+0000 7fc20c869640 1 -- 172.21.15.5:0/2406007395 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2080b62f0 con 0x7fc2080b4d50 2024-03-18T15:33:48.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.597+0000 7f544f7fe640 1 --2- 172.21.15.5:0/1547569341 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5450145570 0x7f54500b4f40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5448009e50 tx=0x7f544801ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2371f8659140670 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.597+0000 7f544e7fc640 1 -- 172.21.15.5:0/1547569341 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54480090d0 con 0x7f5450145570 2024-03-18T15:33:48.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.597+0000 7f544e7fc640 1 -- 172.21.15.5:0/1547569341 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54480a57f0 con 0x7f5450145570 2024-03-18T15:33:48.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.597+0000 7f544e7fc640 1 -- 172.21.15.5:0/1547569341 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5448009270 con 0x7f5450145570 2024-03-18T15:33:48.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.597+0000 7fe558c5b640 1 --2- 172.21.15.5:0/2767311152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5541479a0 0x7fe554147d80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe554141e00 tx=0x7fe54801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=673924b994513c3f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7fa673fff640 1 -- 172.21.15.5:0/3431074685 >> v2:172.21.15.80:3300/0 conn(0x7fa6741479a0 msgr2=0x7fa674147d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:48.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7fa673fff640 1 --2- 172.21.15.5:0/3431074685 >> v2:172.21.15.80:3300/0 conn(0x7fa6741479a0 0x7fa674147d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7fa673fff640 1 -- 172.21.15.5:0/3431074685 >> v1:172.21.15.5:6789/0 conn(0x7fa674071240 legacy=0x7fa67413dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.593+0000 7fa673fff640 1 -- 172.21.15.5:0/3431074685 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa674072a70 con 0x7fa6741482c0 2024-03-18T15:33:48.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.597+0000 7f54561c9640 1 -- 172.21.15.5:0/1547569341 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5450145570 msgr2=0x7f54500b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.597+0000 7f54561c9640 1 --2- 172.21.15.5:0/1547569341 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5450145570 0x7f54500b4f40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5448009e50 tx=0x7f544801ca30 comp rx=0 tx=0).stop 2024-03-18T15:33:48.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.597+0000 7f54561c9640 1 -- 172.21.15.5:0/1547569341 shutdown_connections 2024-03-18T15:33:48.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.597+0000 7f54561c9640 1 --2- 172.21.15.5:0/1547569341 >> v2:172.21.15.5:3300/0 conn(0x7f545014dbd0 0x7f545014ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.597+0000 7f54561c9640 1 --2- 172.21.15.5:0/1547569341 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5450145570 0x7f54500b4f40 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.597+0000 7f54561c9640 1 -- 172.21.15.5:0/1547569341 >> 172.21.15.5:0/1547569341 conn(0x7f54500b5a40 msgr2=0x7f54500b5e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:48.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.597+0000 7f54561c9640 1 -- 172.21.15.5:0/1547569341 shutdown_connections 2024-03-18T15:33:48.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.597+0000 7fa673fff640 1 --2- 172.21.15.5:0/3431074685 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa6741482c0 0x7fa674070d00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa6680097c0 tx=0x7fa66801c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=889f47c9ffc97613 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.597+0000 7f54561c9640 1 -- 172.21.15.5:0/1547569341 wait complete. 2024-03-18T15:33:48.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.597+0000 7fe5537fe640 1 -- 172.21.15.5:0/2767311152 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5480b2070 con 0x7fe5541479a0 2024-03-18T15:33:48.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.597+0000 7fe5537fe640 1 -- 172.21.15.5:0/2767311152 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5480ad640 con 0x7fe5541479a0 2024-03-18T15:33:48.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.597+0000 7fe5537fe640 1 -- 172.21.15.5:0/2767311152 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5480ada10 con 0x7fe5541479a0 2024-03-18T15:33:48.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.597+0000 7fc20c869640 1 --2- 172.21.15.5:0/2406007395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2080b4d50 0x7fc208147cf0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc1f80097f0 tx=0x7fc1f801c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46d2ab37c68433c9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.597+0000 7fc1fffff640 1 -- 172.21.15.5:0/2406007395 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f80b1070 con 0x7fc2080b4d50 2024-03-18T15:33:48.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.597+0000 7fc1fffff640 1 -- 172.21.15.5:0/2406007395 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1f80ac600 con 0x7fc2080b4d50 2024-03-18T15:33:48.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.597+0000 7fc1fffff640 1 -- 172.21.15.5:0/2406007395 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f80ac9d0 con 0x7fc2080b4d50 2024-03-18T15:33:48.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.597+0000 7fe55aebf640 1 -- 172.21.15.5:0/2767311152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5541479a0 msgr2=0x7fe554147d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.597+0000 7fe55aebf640 1 --2- 172.21.15.5:0/2767311152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5541479a0 0x7fe554147d80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe554141e00 tx=0x7fe54801c3d0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.597+0000 7f54561c9640 1 Processor -- start 2024-03-18T15:33:48.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.597+0000 7fe55aebf640 1 -- 172.21.15.5:0/2767311152 shutdown_connections 2024-03-18T15:33:48.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.597+0000 7fe55aebf640 1 --2- 172.21.15.5:0/2767311152 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe554071240 0x7fe55413dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.597+0000 7fe55aebf640 1 --2- 172.21.15.5:0/2767311152 >> v2:172.21.15.80:3300/0 conn(0x7fe5541482c0 0x7fe554070d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.597+0000 7fe55aebf640 1 --2- 172.21.15.5:0/2767311152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5541479a0 0x7fe554147d80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.597+0000 7fe55aebf640 1 -- 172.21.15.5:0/2767311152 >> 172.21.15.5:0/2767311152 conn(0x7fe5540b7a80 msgr2=0x7fe5540b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:48.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.597+0000 7f54561c9640 1 -- start start 2024-03-18T15:33:48.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.597+0000 7fc20f2ce640 1 -- 172.21.15.5:0/2406007395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2080b4d50 msgr2=0x7fc208147cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.597+0000 7fc20f2ce640 1 --2- 172.21.15.5:0/2406007395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2080b4d50 0x7fc208147cf0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc1f80097f0 tx=0x7fc1f801c380 comp rx=0 tx=0).stop 2024-03-18T15:33:48.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.597+0000 7fc20f2ce640 1 -- 172.21.15.5:0/2406007395 shutdown_connections 2024-03-18T15:33:48.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.597+0000 7fc20f2ce640 1 --2- 172.21.15.5:0/2406007395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2080b4d50 0x7fc208147cf0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.597+0000 7fc20f2ce640 1 -- 172.21.15.5:0/2406007395 >> 172.21.15.5:0/2406007395 conn(0x7fc20806e1c0 msgr2=0x7fc20806e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:48.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.597+0000 7fc20f2ce640 1 -- 172.21.15.5:0/2406007395 shutdown_connections 2024-03-18T15:33:48.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.597+0000 7f54561c9640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f54500b5480 0x7f5450071bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.597+0000 7fc20f2ce640 1 -- 172.21.15.5:0/2406007395 wait complete. 2024-03-18T15:33:48.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.597+0000 7f54561c9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5450145570 0x7f54501deef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.597+0000 7f54561c9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f545014dbd0 0x7f54501df430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.597+0000 7f54561c9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f54500b1a50 con 0x7f545014dbd0 2024-03-18T15:33:48.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.597+0000 7f54561c9640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f54500b3390 con 0x7f5450145570 2024-03-18T15:33:48.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.597+0000 7f54561c9640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f5450146030 con 0x7f54500b5480 2024-03-18T15:33:48.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7fe55aebf640 1 -- 172.21.15.5:0/2767311152 shutdown_connections 2024-03-18T15:33:48.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7fe55aebf640 1 -- 172.21.15.5:0/2767311152 wait complete. 2024-03-18T15:33:48.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7f544ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f545014dbd0 0x7f54501df430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7f544ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f545014dbd0 0x7f54501df430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38444/0 (socket says 172.21.15.5:38444) 2024-03-18T15:33:48.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7f544ffff640 1 -- 172.21.15.5:0/2129935095 learned_addr learned my addr 172.21.15.5:0/2129935095 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:48.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7f544f7fe640 1 --2- 172.21.15.5:0/2129935095 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f54500b5480 0x7f5450071bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7f544effd640 1 --2- 172.21.15.5:0/2129935095 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5450145570 0x7f54501deef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7f544ffff640 1 -- 172.21.15.5:0/2129935095 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f54500b5480 msgr2=0x7f5450071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7f544ffff640 1 --2- 172.21.15.5:0/2129935095 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f54500b5480 0x7f5450071bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7f544ffff640 1 -- 172.21.15.5:0/2129935095 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5450145570 msgr2=0x7f54501deef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7f544ffff640 1 --2- 172.21.15.5:0/2129935095 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5450145570 0x7f54501deef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7f544ffff640 1 -- 172.21.15.5:0/2129935095 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f545009b220 con 0x7f545014dbd0 2024-03-18T15:33:48.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7f544ffff640 1 --2- 172.21.15.5:0/2129935095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f545014dbd0 0x7f54501df430 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5444012be0 tx=0x7f5444010490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7f5437fff640 1 -- 172.21.15.5:0/2129935095 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f544400d2d0 con 0x7f545014dbd0 2024-03-18T15:33:48.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7f5437fff640 1 -- 172.21.15.5:0/2129935095 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f544401d070 con 0x7f545014dbd0 2024-03-18T15:33:48.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7f54561c9640 1 -- 172.21.15.5:0/2129935095 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54501de9d0 con 0x7f545014dbd0 2024-03-18T15:33:48.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7f5437fff640 1 -- 172.21.15.5:0/2129935095 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5444010d40 con 0x7f545014dbd0 2024-03-18T15:33:48.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7f54561c9640 1 -- 172.21.15.5:0/2129935095 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5450140240 con 0x7f545014dbd0 2024-03-18T15:33:48.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7fc20f2ce640 1 Processor -- start 2024-03-18T15:33:48.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7fe55aebf640 1 Processor -- start 2024-03-18T15:33:48.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7fe55aebf640 1 -- start start 2024-03-18T15:33:48.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7f54561c9640 1 -- 172.21.15.5:0/2129935095 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f545004f550 con 0x7f545014dbd0 2024-03-18T15:33:48.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7fc20f2ce640 1 -- start start 2024-03-18T15:33:48.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7fc20f2ce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc20806b4e0 0x7fc2080ba1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7fc20f2ce640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc208072d50 0x7fc208144370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7fc20f2ce640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc2081448b0 0x7fc208145510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7fc20f2ce640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc208155820 con 0x7fc20806b4e0 2024-03-18T15:33:48.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7fc20f2ce640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc2081556a0 con 0x7fc208072d50 2024-03-18T15:33:48.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7fc20f2ce640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fc2081559a0 con 0x7fc2081448b0 2024-03-18T15:33:48.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7fc20c869640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc208072d50 0x7fc208144370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7fc20c869640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc208072d50 0x7fc208144370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:58958/0 (socket says 172.21.15.5:58958) 2024-03-18T15:33:48.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7fc20c869640 1 -- 172.21.15.5:0/1842036320 learned_addr learned my addr 172.21.15.5:0/1842036320 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:48.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7fc20d06a640 1 --2- 172.21.15.5:0/1842036320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc20806b4e0 0x7fc2080ba1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7fe55aebf640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe554071240 0x7fe554146b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7fe55aebf640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe5541479a0 0x7fe554140800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7fc20d86b640 1 --2- 172.21.15.5:0/1842036320 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc2081448b0 0x7fc208145510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7fe55aebf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe55413f460 0x7fe554140d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7fc20c869640 1 -- 172.21.15.5:0/1842036320 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc2081448b0 msgr2=0x7fc208145510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7fc20c869640 1 --2- 172.21.15.5:0/1842036320 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc2081448b0 0x7fc208145510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7fc20c869640 1 -- 172.21.15.5:0/1842036320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc20806b4e0 msgr2=0x7fc2080ba1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7fc20c869640 1 --2- 172.21.15.5:0/1842036320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc20806b4e0 0x7fc2080ba1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7fc20c869640 1 -- 172.21.15.5:0/1842036320 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc208144da0 con 0x7fc208072d50 2024-03-18T15:33:48.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7fe55aebf640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe554144b60 con 0x7fe55413f460 2024-03-18T15:33:48.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7fe55aebf640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe5541449e0 con 0x7fe554071240 2024-03-18T15:33:48.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7fe553fff640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe5541479a0 0x7fe554140800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7fe553fff640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe5541479a0 0x7fe554140800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:52888/0 (socket says 172.21.15.5:52888) 2024-03-18T15:33:48.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7fe553fff640 1 -- 172.21.15.5:0/2811507121 learned_addr learned my addr 172.21.15.5:0/2811507121 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:48.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7fe558c5b640 1 --2- 172.21.15.5:0/2811507121 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe554071240 0x7fe554146b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7fe55945c640 1 --2- 172.21.15.5:0/2811507121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe55413f460 0x7fe554140d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7fe55aebf640 1 -- 172.21.15.5:0/2811507121 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fe554144ce0 con 0x7fe5541479a0 2024-03-18T15:33:48.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7fc20c869640 1 --2- 172.21.15.5:0/1842036320 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc208072d50 0x7fc208144370 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc1f80af040 tx=0x7fc1f801abc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7f5437fff640 1 -- 172.21.15.5:0/2129935095 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f5444016050 con 0x7f545014dbd0 2024-03-18T15:33:48.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7fe55945c640 1 -- 172.21.15.5:0/2811507121 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe5541479a0 msgr2=0x7fe554140800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7fe55945c640 1 --2- 172.21.15.5:0/2811507121 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe5541479a0 0x7fe554140800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7fe55945c640 1 -- 172.21.15.5:0/2811507121 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe554071240 msgr2=0x7fe554146b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7fe55945c640 1 --2- 172.21.15.5:0/2811507121 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe554071240 0x7fe554146b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7fe55945c640 1 -- 172.21.15.5:0/2811507121 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe55413f940 con 0x7fe55413f460 2024-03-18T15:33:48.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7fc1fdffb640 1 -- 172.21.15.5:0/1842036320 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f80bb020 con 0x7fc208072d50 2024-03-18T15:33:48.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7fc1fdffb640 1 -- 172.21.15.5:0/1842036320 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1f80b7420 con 0x7fc208072d50 2024-03-18T15:33:48.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7fc1fdffb640 1 -- 172.21.15.5:0/1842036320 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f80b1040 con 0x7fc208072d50 2024-03-18T15:33:48.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7fc20f2ce640 1 -- 172.21.15.5:0/1842036320 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc208143450 con 0x7fc208072d50 2024-03-18T15:33:48.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.601+0000 7fc20f2ce640 1 -- 172.21.15.5:0/1842036320 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc208143f50 con 0x7fc208072d50 2024-03-18T15:33:48.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.605+0000 7f5437fff640 1 --2- 172.21.15.5:0/2129935095 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f543c0676b0 0x7f543c069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.605+0000 7fe55945c640 1 --2- 172.21.15.5:0/2811507121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe55413f460 0x7fe554140d40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe54c007d40 tx=0x7fe54c00f6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.605+0000 7f5437fff640 1 -- 172.21.15.5:0/2129935095 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f544408a360 con 0x7f545014dbd0 2024-03-18T15:33:48.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.605+0000 7f544f7fe640 1 --2- 172.21.15.5:0/2129935095 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f543c0676b0 0x7f543c069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.605+0000 7f5437fff640 1 --2- 172.21.15.5:0/2129935095 >> [v2:172.21.15.5:6816/502968448,v1:172.21.15.5:6817/502968448] conn(0x7f543c06f930 0x7f543c071d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.605+0000 7f5437fff640 1 -- 172.21.15.5:0/2129935095 --> [v2:172.21.15.5:6816/502968448,v1:172.21.15.5:6817/502968448] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f543c072400 con 0x7f543c06f930 2024-03-18T15:33:48.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.605+0000 7f5437fff640 1 -- 172.21.15.5:0/2129935095 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f544408a840 con 0x7f545014dbd0 2024-03-18T15:33:48.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.605+0000 7f544effd640 1 --2- 172.21.15.5:0/2129935095 >> [v2:172.21.15.5:6816/502968448,v1:172.21.15.5:6817/502968448] conn(0x7f543c06f930 0x7f543c071d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.605+0000 7fc20f2ce640 1 -- 172.21.15.5:0/1842036320 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc20804f550 con 0x7fc208072d50 2024-03-18T15:33:48.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.605+0000 7f544effd640 1 --2- 172.21.15.5:0/2129935095 >> [v2:172.21.15.5:6816/502968448,v1:172.21.15.5:6817/502968448] conn(0x7f543c06f930 0x7f543c071d50 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.605+0000 7f544f7fe640 1 --2- 172.21.15.5:0/2129935095 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f543c0676b0 0x7f543c069b70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f54480ae000 tx=0x7f54480a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.605+0000 7fe5517fa640 1 -- 172.21.15.5:0/2811507121 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54c016070 con 0x7fe55413f460 2024-03-18T15:33:48.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.605+0000 7fe5517fa640 1 -- 172.21.15.5:0/2811507121 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe54c003be0 con 0x7fe55413f460 2024-03-18T15:33:48.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.605+0000 7fe5517fa640 1 -- 172.21.15.5:0/2811507121 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54c004020 con 0x7fe55413f460 2024-03-18T15:33:48.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.605+0000 7fe55aebf640 1 -- 172.21.15.5:0/2811507121 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe554145330 con 0x7fe55413f460 2024-03-18T15:33:48.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.605+0000 7fe55aebf640 1 -- 172.21.15.5:0/2811507121 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe554145980 con 0x7fe55413f460 2024-03-18T15:33:48.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.605+0000 7fe55aebf640 1 -- 172.21.15.5:0/2811507121 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe534001040 con 0x7fe55413f460 2024-03-18T15:33:48.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.605+0000 7f5437fff640 1 -- 172.21.15.5:0/2129935095 <== osd.2 v2:172.21.15.5:6816/502968448 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f543c072400 con 0x7f543c06f930 2024-03-18T15:33:48.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.605+0000 7fc1fdffb640 1 -- 172.21.15.5:0/1842036320 <== mon.1 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fc1f80be020 con 0x7fc208072d50 2024-03-18T15:33:48.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.605+0000 7fc1fdffb640 1 --2- 172.21.15.5:0/1842036320 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fc1ec0676b0 0x7fc1ec069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.605+0000 7fc1fdffb640 1 -- 172.21.15.5:0/1842036320 <== mon.1 v2:172.21.15.80:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc1f811fe40 con 0x7fc208072d50 2024-03-18T15:33:48.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.605+0000 7fc1fdffb640 1 --2- 172.21.15.5:0/1842036320 >> [v2:172.21.15.5:6824/2425932735,v1:172.21.15.5:6825/2425932735] conn(0x7fc1ec06f8d0 0x7fc1ec071cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.605+0000 7fc1fdffb640 1 -- 172.21.15.5:0/1842036320 --> [v2:172.21.15.5:6824/2425932735,v1:172.21.15.5:6825/2425932735] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc1f8001d40 con 0x7fc1ec06f8d0 2024-03-18T15:33:48.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.605+0000 7fc1fdffb640 1 -- 172.21.15.5:0/1842036320 <== mon.1 v2:172.21.15.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc1f81201f0 con 0x7fc208072d50 2024-03-18T15:33:48.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.605+0000 7fc20d06a640 1 --2- 172.21.15.5:0/1842036320 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fc1ec0676b0 0x7fc1ec069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.605+0000 7fc20d86b640 1 --2- 172.21.15.5:0/1842036320 >> [v2:172.21.15.5:6824/2425932735,v1:172.21.15.5:6825/2425932735] conn(0x7fc1ec06f8d0 0x7fc1ec071cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.605+0000 7fc20d06a640 1 --2- 172.21.15.5:0/1842036320 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fc1ec0676b0 0x7fc1ec069b70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc204001040 tx=0x7fc204009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.605+0000 7fe5517fa640 1 -- 172.21.15.5:0/2811507121 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fe54c0041c0 con 0x7fe55413f460 2024-03-18T15:33:48.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.609+0000 7fc20d86b640 1 --2- 172.21.15.5:0/1842036320 >> [v2:172.21.15.5:6824/2425932735,v1:172.21.15.5:6825/2425932735] conn(0x7fc1ec06f8d0 0x7fc1ec071cf0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.609+0000 7fe5517fa640 1 --2- 172.21.15.5:0/2811507121 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe538067730 0x7fe538069bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.609+0000 7fe5517fa640 1 -- 172.21.15.5:0/2811507121 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe54c086a10 con 0x7fe55413f460 2024-03-18T15:33:48.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.609+0000 7fe558c5b640 1 --2- 172.21.15.5:0/2811507121 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe538067730 0x7fe538069bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.609+0000 7fe5517fa640 1 --2- 172.21.15.5:0/2811507121 >> [v2:172.21.15.80:6824/627153447,v1:172.21.15.80:6825/627153447] conn(0x7fe53806f950 0x7fe538071d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.609+0000 7fe5517fa640 1 -- 172.21.15.5:0/2811507121 --> [v2:172.21.15.80:6824/627153447,v1:172.21.15.80:6825/627153447] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe54c01ce50 con 0x7fe53806f950 2024-03-18T15:33:48.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.609+0000 7fe5517fa640 1 -- 172.21.15.5:0/2811507121 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe54c086e00 con 0x7fe55413f460 2024-03-18T15:33:48.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.609+0000 7fe553fff640 1 --2- 172.21.15.5:0/2811507121 >> [v2:172.21.15.80:6824/627153447,v1:172.21.15.80:6825/627153447] conn(0x7fe53806f950 0x7fe538071d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.609+0000 7fe558c5b640 1 --2- 172.21.15.5:0/2811507121 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe538067730 0x7fe538069bf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe5480af000 tx=0x7fe5480b7040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.609+0000 7fa6737fe640 1 -- 172.21.15.5:0/3431074685 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6680b0070 con 0x7fa6741482c0 2024-03-18T15:33:48.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.609+0000 7fa6737fe640 1 -- 172.21.15.5:0/3431074685 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6680ab600 con 0x7fa6741482c0 2024-03-18T15:33:48.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.609+0000 7fa6737fe640 1 -- 172.21.15.5:0/3431074685 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6680ab9b0 con 0x7fa6741482c0 2024-03-18T15:33:48.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.609+0000 7fa67afe1640 1 -- 172.21.15.5:0/3431074685 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa6741482c0 msgr2=0x7fa674070d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.609+0000 7fa67afe1640 1 --2- 172.21.15.5:0/3431074685 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa6741482c0 0x7fa674070d00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa6680097c0 tx=0x7fa66801c3a0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.609+0000 7fc1fdffb640 1 -- 172.21.15.5:0/1842036320 <== osd.3 v2:172.21.15.5:6824/2425932735 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc1f8001d40 con 0x7fc1ec06f8d0 2024-03-18T15:33:48.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.609+0000 7fa67afe1640 1 -- 172.21.15.5:0/3431074685 shutdown_connections 2024-03-18T15:33:48.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.609+0000 7fa67afe1640 1 --2- 172.21.15.5:0/3431074685 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa6741482c0 0x7fa674070d00 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.609+0000 7fa67afe1640 1 --2- 172.21.15.5:0/3431074685 >> v2:172.21.15.80:3300/0 conn(0x7fa6741479a0 0x7fa674147d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.609+0000 7fa67afe1640 1 -- 172.21.15.5:0/3431074685 >> 172.21.15.5:0/3431074685 conn(0x7fa6740b7a80 msgr2=0x7fa6740b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:48.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.609+0000 7fa67afe1640 1 -- 172.21.15.5:0/3431074685 shutdown_connections 2024-03-18T15:33:48.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.609+0000 7f6e08182640 1 Processor -- start 2024-03-18T15:33:48.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.609+0000 7fa67afe1640 1 -- 172.21.15.5:0/3431074685 wait complete. 2024-03-18T15:33:48.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.609+0000 7fa67afe1640 1 Processor -- start 2024-03-18T15:33:48.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.609+0000 7f6e08182640 1 -- start start 2024-03-18T15:33:48.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.609+0000 7fa67afe1640 1 -- start start 2024-03-18T15:33:48.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.609+0000 7f6e08182640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6e0006b4e0 0x7f6e00071e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.609+0000 7f6e08182640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f6e000711a0 0x7f6e00148960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.609+0000 7f6e08182640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6e00148ea0 0x7f6e0014ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.609+0000 7f6e08182640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f6e00149330 con 0x7f6e00148ea0 2024-03-18T15:33:48.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.609+0000 7f6e08182640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f6e0006b8c0 con 0x7f6e000711a0 2024-03-18T15:33:48.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.609+0000 7f6e08182640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f6e0014b8d0 con 0x7f6e0006b4e0 2024-03-18T15:33:48.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.609+0000 7f6e05f1e640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6e0006b4e0 0x7f6e00071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.609+0000 7f6e05f1e640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6e0006b4e0 0x7f6e00071e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:52892/0 (socket says 172.21.15.5:52892) 2024-03-18T15:33:48.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.609+0000 7fa67afe1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa674071240 0x7fa6741e7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.609+0000 7fa67afe1640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa6741479a0 0x7fa674142ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.609+0000 7fa67afe1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa67413d210 0x7fa674143ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.609+0000 7fa67afe1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa674151620 con 0x7fa674071240 2024-03-18T15:33:48.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.609+0000 7fa67afe1640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa6741514a0 con 0x7fa67413d210 2024-03-18T15:33:48.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.609+0000 7fa67afe1640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fa6741517a0 con 0x7fa6741479a0 2024-03-18T15:33:48.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.609+0000 7fa678d7d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa674071240 0x7fa6741e7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.609+0000 7fa678d7d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa674071240 0x7fa6741e7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38490/0 (socket says 172.21.15.5:38490) 2024-03-18T15:33:48.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.609+0000 7fa678d7d640 1 -- 172.21.15.5:0/3795426679 learned_addr learned my addr 172.21.15.5:0/3795426679 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:48.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.609+0000 7f6e05f1e640 1 -- 172.21.15.5:0/3474155842 learned_addr learned my addr 172.21.15.5:0/3474155842 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:48.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.613+0000 7fa673fff640 1 --2- 172.21.15.5:0/3795426679 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa6741479a0 0x7fa674142ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.613+0000 7fe553fff640 1 --2- 172.21.15.5:0/2811507121 >> [v2:172.21.15.80:6824/627153447,v1:172.21.15.80:6825/627153447] conn(0x7fe53806f950 0x7fe538071d70 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.609+0000 7f6e0671f640 1 --2- 172.21.15.5:0/3474155842 >> v2:172.21.15.5:3300/0 conn(0x7f6e00148ea0 0x7f6e0014ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.613+0000 7fa67957e640 1 --2- 172.21.15.5:0/3795426679 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa67413d210 0x7fa674143ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.613+0000 7fa678d7d640 1 -- 172.21.15.5:0/3795426679 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa6741479a0 msgr2=0x7fa674142ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.613+0000 7fa678d7d640 1 --2- 172.21.15.5:0/3795426679 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa6741479a0 0x7fa674142ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.613+0000 7fa678d7d640 1 -- 172.21.15.5:0/3795426679 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa67413d210 msgr2=0x7fa674143ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.613+0000 7fa678d7d640 1 --2- 172.21.15.5:0/3795426679 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa67413d210 0x7fa674143ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.613+0000 7fa678d7d640 1 -- 172.21.15.5:0/3795426679 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6741448b0 con 0x7fa674071240 2024-03-18T15:33:48.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.613+0000 7f6e05f1e640 1 -- 172.21.15.5:0/3474155842 >> v2:172.21.15.80:3300/0 conn(0x7f6e000711a0 msgr2=0x7f6e00148960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T15:33:48.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.613+0000 7f6e05f1e640 1 --2- 172.21.15.5:0/3474155842 >> v2:172.21.15.80:3300/0 conn(0x7f6e000711a0 0x7f6e00148960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.613+0000 7f6e05f1e640 1 -- 172.21.15.5:0/3474155842 >> v2:172.21.15.5:3300/0 conn(0x7f6e00148ea0 msgr2=0x7f6e0014ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:48.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.613+0000 7f6e05f1e640 1 --2- 172.21.15.5:0/3474155842 >> v2:172.21.15.5:3300/0 conn(0x7f6e00148ea0 0x7f6e0014ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.613+0000 7f6e05f1e640 1 -- 172.21.15.5:0/3474155842 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e000716d0 con 0x7f6e0006b4e0 2024-03-18T15:33:48.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.613+0000 7fa678d7d640 1 --2- 172.21.15.5:0/3795426679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa674071240 0x7fa6741e7cb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa66000e930 tx=0x7fa66000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.613+0000 7fa6717fa640 1 -- 172.21.15.5:0/3795426679 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa660017070 con 0x7fa674071240 2024-03-18T15:33:48.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.613+0000 7fe5517fa640 1 -- 172.21.15.5:0/2811507121 <== osd.5 v2:172.21.15.80:6824/627153447 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe54c01ce50 con 0x7fe53806f950 2024-03-18T15:33:48.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.613+0000 7fa67afe1640 1 -- 172.21.15.5:0/3795426679 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa674144b90 con 0x7fa674071240 2024-03-18T15:33:48.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.613+0000 7fa67afe1640 1 -- 172.21.15.5:0/3795426679 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa674143a00 con 0x7fa674071240 2024-03-18T15:33:48.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.613+0000 7fa67afe1640 1 -- 172.21.15.5:0/3795426679 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa674152be0 con 0x7fa674071240 2024-03-18T15:33:48.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.613+0000 7fa6717fa640 1 -- 172.21.15.5:0/3795426679 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa66000ca00 con 0x7fa674071240 2024-03-18T15:33:48.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.617+0000 7fa6717fa640 1 -- 172.21.15.5:0/3795426679 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66000cdb0 con 0x7fa674071240 2024-03-18T15:33:48.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.617+0000 7f6e05f1e640 1 --2- 172.21.15.5:0/3474155842 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6e0006b4e0 0x7f6e00071e40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6e00074f60 tx=0x7f6df801c690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed5fa8a690f7b940 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.617+0000 7fa6717fa640 1 -- 172.21.15.5:0/3795426679 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fa66001a070 con 0x7fa674071240 2024-03-18T15:33:48.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.617+0000 7f6e04f1c640 1 -- 172.21.15.5:0/3474155842 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df80a5510 con 0x7f6e0006b4e0 2024-03-18T15:33:48.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.617+0000 7f6e04f1c640 1 -- 172.21.15.5:0/3474155842 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6df80a56b0 con 0x7f6e0006b4e0 2024-03-18T15:33:48.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.617+0000 7f6e04f1c640 1 -- 172.21.15.5:0/3474155842 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df80a5a80 con 0x7f6e0006b4e0 2024-03-18T15:33:48.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.617+0000 7f6e08182640 1 -- 172.21.15.5:0/3474155842 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6e0006b4e0 msgr2=0x7f6e00071e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.617+0000 7f6e08182640 1 --2- 172.21.15.5:0/3474155842 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6e0006b4e0 0x7f6e00071e40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6e00074f60 tx=0x7f6df801c690 comp rx=0 tx=0).stop 2024-03-18T15:33:48.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.617+0000 7f6e08182640 1 -- 172.21.15.5:0/3474155842 shutdown_connections 2024-03-18T15:33:48.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.617+0000 7f6e08182640 1 --2- 172.21.15.5:0/3474155842 >> v2:172.21.15.5:3300/0 conn(0x7f6e00148ea0 0x7f6e0014ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.617+0000 7f6e08182640 1 --2- 172.21.15.5:0/3474155842 >> v2:172.21.15.80:3300/0 conn(0x7f6e000711a0 0x7f6e00148960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.617+0000 7f6e08182640 1 --2- 172.21.15.5:0/3474155842 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6e0006b4e0 0x7f6e00071e40 secure :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6e00074f60 tx=0x7f6df801c690 comp rx=0 tx=0).stop 2024-03-18T15:33:48.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.617+0000 7f6e08182640 1 -- 172.21.15.5:0/3474155842 >> 172.21.15.5:0/3474155842 conn(0x7f6e000b7a80 msgr2=0x7f6e000b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:48.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.617+0000 7f6e08182640 1 -- 172.21.15.5:0/3474155842 shutdown_connections 2024-03-18T15:33:48.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.617+0000 7f6e08182640 1 -- 172.21.15.5:0/3474155842 wait complete. 2024-03-18T15:33:48.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.617+0000 7f6e08182640 1 Processor -- start 2024-03-18T15:33:48.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.621+0000 7f4297fff640 1 Processor -- start 2024-03-18T15:33:48.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.621+0000 7f4297fff640 1 -- start start 2024-03-18T15:33:48.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.621+0000 7f4297fff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f42900c5ea0 0x7f42900c6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.621+0000 7f4297fff640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f42900d1480 0x7f42900d3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.621+0000 7f4297fff640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42900c6890 con 0x7f42900cc3c0 2024-03-18T15:33:48.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.621+0000 7f4297fff640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f42900c8e20 con 0x7f42900c5ea0 2024-03-18T15:33:48.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.621+0000 7f4297fff640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f429001e460 con 0x7f42900d1480 2024-03-18T15:33:48.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.621+0000 7fa6717fa640 1 --2- 172.21.15.5:0/3795426679 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa658067760 0x7fa658069c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.621+0000 7fa6717fa640 1 -- 172.21.15.5:0/3795426679 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa660086170 con 0x7fa674071240 2024-03-18T15:33:48.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.621+0000 7fa6717fa640 1 --2- 172.21.15.5:0/3795426679 >> [v2:172.21.15.80:6803/488342527,v1:172.21.15.80:6805/488342527] conn(0x7fa65806f9e0 0x7fa658071e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.621+0000 7fa673fff640 1 --2- 172.21.15.5:0/3795426679 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa658067760 0x7fa658069c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.621+0000 7fa6717fa640 1 -- 172.21.15.5:0/3795426679 --> [v2:172.21.15.80:6803/488342527,v1:172.21.15.80:6805/488342527] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa66000ca00 con 0x7fa65806f9e0 2024-03-18T15:33:48.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.621+0000 7fa6717fa640 1 -- 172.21.15.5:0/3795426679 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa660086490 con 0x7fa674071240 2024-03-18T15:33:48.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.621+0000 7fa67957e640 1 --2- 172.21.15.5:0/3795426679 >> [v2:172.21.15.80:6803/488342527,v1:172.21.15.80:6805/488342527] conn(0x7fa65806f9e0 0x7fa658071e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.621+0000 7f6e08182640 1 -- start start 2024-03-18T15:33:48.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.621+0000 7f6e08182640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6e000711a0 0x7f6e001502f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.621+0000 7f6e08182640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6e00148ea0 0x7f6e001509f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.621+0000 7f6e08182640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e000b3440 0x7f6e0013ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.621+0000 7f6e08182640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0014c670 con 0x7f6e000b3440 2024-03-18T15:33:48.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.621+0000 7f6e08182640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f6e0014c150 con 0x7f6e00148ea0 2024-03-18T15:33:48.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.621+0000 7f6e08182640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f6e000b24a0 con 0x7f6e000711a0 2024-03-18T15:33:48.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.621+0000 7f42977fe640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f42900d1480 0x7f42900d3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.621+0000 7f42977fe640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f42900d1480 0x7f42900d3870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:52896/0 (socket says 172.21.15.5:52896) 2024-03-18T15:33:48.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.621+0000 7f42977fe640 1 -- 172.21.15.5:0/4049034684 learned_addr learned my addr 172.21.15.5:0/4049034684 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:48.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.621+0000 7f42967fc640 1 --2- 172.21.15.5:0/4049034684 >> v2:172.21.15.80:3300/0 conn(0x7f42900c5ea0 0x7f42900c6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.621+0000 7f6e0671f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e000b3440 0x7f6e0013ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.621+0000 7f6e0671f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e000b3440 0x7f6e0013ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38506/0 (socket says 172.21.15.5:38506) 2024-03-18T15:33:48.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.621+0000 7f6e0671f640 1 -- 172.21.15.5:0/2182030469 learned_addr learned my addr 172.21.15.5:0/2182030469 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:48.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.621+0000 7f6e0571d640 1 --2- 172.21.15.5:0/2182030469 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6e00148ea0 0x7f6e001509f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.621+0000 7fa67957e640 1 --2- 172.21.15.5:0/3795426679 >> [v2:172.21.15.80:6803/488342527,v1:172.21.15.80:6805/488342527] conn(0x7fa65806f9e0 0x7fa658071e00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.621+0000 7f6e05f1e640 1 --2- 172.21.15.5:0/2182030469 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6e000711a0 0x7f6e001502f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.621+0000 7f42977fe640 1 -- 172.21.15.5:0/4049034684 >> v2:172.21.15.80:3300/0 conn(0x7f42900c5ea0 msgr2=0x7f42900c6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:48.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.621+0000 7f42977fe640 1 --2- 172.21.15.5:0/4049034684 >> v2:172.21.15.80:3300/0 conn(0x7f42900c5ea0 0x7f42900c6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.621+0000 7f42977fe640 1 -- 172.21.15.5:0/4049034684 >> v1:172.21.15.80:6789/0 conn(0x7f42900cc3c0 legacy=0x7f42900c92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.621+0000 7fa673fff640 1 --2- 172.21.15.5:0/3795426679 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa658067760 0x7fa658069c20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa6680ad000 tx=0x7fa6680a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.621+0000 7f6e0671f640 1 -- 172.21.15.5:0/2182030469 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6e000711a0 msgr2=0x7f6e001502f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.621+0000 7f6e0671f640 1 --2- 172.21.15.5:0/2182030469 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6e000711a0 0x7f6e001502f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.621+0000 7f6e0671f640 1 -- 172.21.15.5:0/2182030469 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6e00148ea0 msgr2=0x7f6e001509f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.621+0000 7f6e0671f640 1 --2- 172.21.15.5:0/2182030469 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6e00148ea0 0x7f6e001509f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.621+0000 7f6e0671f640 1 -- 172.21.15.5:0/2182030469 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e00151d80 con 0x7f6e000b3440 2024-03-18T15:33:48.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.621+0000 7f6e0671f640 1 --2- 172.21.15.5:0/2182030469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e000b3440 0x7f6e0013ca00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc00e7f0 tx=0x7f6dfc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.621+0000 7f42977fe640 1 -- 172.21.15.5:0/4049034684 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42900cc7a0 con 0x7f42900d1480 2024-03-18T15:33:48.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.621+0000 7f42967fc640 1 --2- 172.21.15.5:0/4049034684 >> v2:172.21.15.80:3300/0 conn(0x7f42900c5ea0 0x7f42900c6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:33:48.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.625+0000 7f42977fe640 1 --2- 172.21.15.5:0/4049034684 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f42900d1480 0x7f42900d3870 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f429806a8d0 tx=0x7f4298098d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9986062575e5605c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.625+0000 7f54561c9640 1 -- 172.21.15.5:0/2129935095 --> [v2:172.21.15.5:6816/502968448,v1:172.21.15.5:6817/502968448] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f54500b6de0 con 0x7f543c06f930 2024-03-18T15:33:48.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.621+0000 7fa6717fa640 1 -- 172.21.15.5:0/3795426679 <== osd.6 v2:172.21.15.80:6803/488342527 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa66000ca00 con 0x7fa65806f9e0 2024-03-18T15:33:48.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.625+0000 7f5437fff640 1 -- 172.21.15.5:0/2129935095 <== osd.2 v2:172.21.15.5:6816/502968448 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f54500b6de0 con 0x7f543c06f930 2024-03-18T15:33:48.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.625+0000 7f54561c9640 1 -- 172.21.15.5:0/2129935095 >> [v2:172.21.15.5:6816/502968448,v1:172.21.15.5:6817/502968448] conn(0x7f543c06f930 msgr2=0x7f543c071d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.625+0000 7f54561c9640 1 --2- 172.21.15.5:0/2129935095 >> [v2:172.21.15.5:6816/502968448,v1:172.21.15.5:6817/502968448] conn(0x7f543c06f930 0x7f543c071d50 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.625+0000 7f54561c9640 1 -- 172.21.15.5:0/2129935095 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f543c0676b0 msgr2=0x7f543c069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.625+0000 7f54561c9640 1 --2- 172.21.15.5:0/2129935095 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f543c0676b0 0x7f543c069b70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f54480ae000 tx=0x7f54480a7070 comp rx=0 tx=0).stop 2024-03-18T15:33:48.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.625+0000 7f54561c9640 1 -- 172.21.15.5:0/2129935095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f545014dbd0 msgr2=0x7f54501df430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.625+0000 7f54561c9640 1 --2- 172.21.15.5:0/2129935095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f545014dbd0 0x7f54501df430 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5444012be0 tx=0x7f5444010490 comp rx=0 tx=0).stop 2024-03-18T15:33:48.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.625+0000 7f544ffff640 1 -- 172.21.15.5:0/2129935095 reap_dead start 2024-03-18T15:33:48.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.625+0000 7f54561c9640 1 -- 172.21.15.5:0/2129935095 shutdown_connections 2024-03-18T15:33:48.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.625+0000 7f54561c9640 1 -- 172.21.15.5:0/2129935095 >> 172.21.15.5:0/2129935095 conn(0x7f54500b5a40 msgr2=0x7f54501446b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:48.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.625+0000 7f54561c9640 1 -- 172.21.15.5:0/2129935095 shutdown_connections 2024-03-18T15:33:48.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.625+0000 7f54561c9640 1 -- 172.21.15.5:0/2129935095 wait complete. 2024-03-18T15:33:48.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.621+0000 7f6df67fc640 1 -- 172.21.15.5:0/2182030469 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc017070 con 0x7f6e000b3440 2024-03-18T15:33:48.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.621+0000 7f6df67fc640 1 -- 172.21.15.5:0/2182030469 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dfc00c940 con 0x7f6e000b3440 2024-03-18T15:33:48.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.621+0000 7f6df67fc640 1 -- 172.21.15.5:0/2182030469 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc00eea0 con 0x7f6e000b3440 2024-03-18T15:33:48.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.621+0000 7f6e08182640 1 -- 172.21.15.5:0/2182030469 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e000b2130 con 0x7f6e000b3440 2024-03-18T15:33:48.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.621+0000 7f6e08182640 1 -- 172.21.15.5:0/2182030469 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e00150f30 con 0x7f6e000b3440 2024-03-18T15:33:48.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.625+0000 7f6e05f1e640 1 --2- 172.21.15.5:0/2182030469 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6e000711a0 0x7f6e001502f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:33:48.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.625+0000 7f6e08182640 1 -- 172.21.15.5:0/2182030469 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6e0004f550 con 0x7f6e000b3440 2024-03-18T15:33:48.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.625+0000 7f090eae2640 1 Processor -- start 2024-03-18T15:33:48.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.625+0000 7f4295ffb640 1 -- 172.21.15.5:0/4049034684 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42981226f0 con 0x7f42900d1480 2024-03-18T15:33:48.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.625+0000 7f4295ffb640 1 -- 172.21.15.5:0/4049034684 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4298122890 con 0x7f42900d1480 2024-03-18T15:33:48.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.625+0000 7f090eae2640 1 -- start start 2024-03-18T15:33:48.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.629+0000 7f4297fff640 1 -- 172.21.15.5:0/4049034684 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f42900d1480 msgr2=0x7f42900d3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.629+0000 7f4297fff640 1 --2- 172.21.15.5:0/4049034684 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f42900d1480 0x7f42900d3870 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f429806a8d0 tx=0x7f4298098d80 comp rx=0 tx=0).stop 2024-03-18T15:33:48.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.629+0000 7f090eae2640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f09000c47c0 0x7f0900039770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.629+0000 7f090eae2640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09000cb220 con 0x7f0900037340 2024-03-18T15:33:48.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.629+0000 7f090eae2640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0900037720 con 0x7f0900038e10 2024-03-18T15:33:48.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.629+0000 7f090eae2640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f0900039dd0 con 0x7f09000c47c0 2024-03-18T15:33:48.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.629+0000 7f4297fff640 1 -- 172.21.15.5:0/4049034684 shutdown_connections 2024-03-18T15:33:48.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.629+0000 7f4297fff640 1 --2- 172.21.15.5:0/4049034684 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f42900d1480 0x7f42900d3870 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.629+0000 7f4297fff640 1 --2- 172.21.15.5:0/4049034684 >> v2:172.21.15.80:3300/0 conn(0x7f42900c5ea0 0x7f42900c6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.629+0000 7f4297fff640 1 -- 172.21.15.5:0/4049034684 >> 172.21.15.5:0/4049034684 conn(0x7f429000ab50 msgr2=0x7f4290006b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:48.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.629+0000 7f090e2e1640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f09000c47c0 0x7f0900039770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.629+0000 7f6df67fc640 1 -- 172.21.15.5:0/2182030469 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f6dfc01a070 con 0x7f6e000b3440 2024-03-18T15:33:48.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.629+0000 7f6df67fc640 1 --2- 172.21.15.5:0/2182030469 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6de4067660 0x7f6de4069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.629+0000 7f6df67fc640 1 -- 172.21.15.5:0/2182030469 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f6dfc085a50 con 0x7f6e000b3440 2024-03-18T15:33:48.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.629+0000 7f6df67fc640 1 --2- 172.21.15.5:0/2182030469 >> [v2:172.21.15.80:6800/982831666,v1:172.21.15.80:6801/982831666] conn(0x7f6de406f880 0x7f6de4071ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.629+0000 7f6df67fc640 1 -- 172.21.15.5:0/2182030469 --> [v2:172.21.15.80:6800/982831666,v1:172.21.15.80:6801/982831666] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6de4072350 con 0x7f6de406f880 2024-03-18T15:33:48.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.629+0000 7f6df67fc640 1 -- 172.21.15.5:0/2182030469 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f6dfc08a2a0 con 0x7f6e000b3440 2024-03-18T15:33:48.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.629+0000 7f090e2e1640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f09000c47c0 0x7f0900039770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:59000/0 (socket says 172.21.15.5:59000) 2024-03-18T15:33:48.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.629+0000 7f090e2e1640 1 -- 172.21.15.5:0/3535181337 learned_addr learned my addr 172.21.15.5:0/3535181337 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:48.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.629+0000 7f1f6b2bf640 1 -- 172.21.15.5:0/1994964951 --> [v2:172.21.15.5:6800/3901761126,v1:172.21.15.5:6801/3901761126] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1f44002de0 con 0x7f1f4806f910 2024-03-18T15:33:48.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.629+0000 7f4297fff640 1 -- 172.21.15.5:0/4049034684 shutdown_connections 2024-03-18T15:33:48.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.629+0000 7f4297fff640 1 -- 172.21.15.5:0/4049034684 wait complete. 2024-03-18T15:33:48.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.629+0000 7f1f59ffb640 1 -- 172.21.15.5:0/1994964951 <== osd.1 v2:172.21.15.5:6800/3901761126 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1f44002de0 con 0x7f1f4806f910 2024-03-18T15:33:48.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.629+0000 7f090e2e1640 1 -- 172.21.15.5:0/3535181337 >> v1:172.21.15.80:6789/0 conn(0x7f0900038e10 legacy=0x7f09000c40b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-18T15:33:48.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.629+0000 7f090cade640 1 -- 172.21.15.5:0/3535181337 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1863206963 0 0) 0x7f09000cb220 con 0x7f0900037340 2024-03-18T15:33:48.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.629+0000 7f090e2e1640 1 -- 172.21.15.5:0/3535181337 >> v1:172.21.15.5:6789/0 conn(0x7f0900037340 legacy=0x7f09000cab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.629+0000 7f090e2e1640 1 -- 172.21.15.5:0/3535181337 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0900037d20 con 0x7f09000c47c0 2024-03-18T15:33:48.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.629+0000 7f6e0571d640 1 --2- 172.21.15.5:0/2182030469 >> [v2:172.21.15.80:6800/982831666,v1:172.21.15.80:6801/982831666] conn(0x7f6de406f880 0x7f6de4071ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.629+0000 7f6e05f1e640 1 --2- 172.21.15.5:0/2182030469 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6de4067660 0x7f6de4069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.629+0000 7f6e0571d640 1 --2- 172.21.15.5:0/2182030469 >> [v2:172.21.15.80:6800/982831666,v1:172.21.15.80:6801/982831666] conn(0x7f6de406f880 0x7f6de4071ca0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.629+0000 7f1f6b2bf640 1 -- 172.21.15.5:0/1994964951 >> [v2:172.21.15.5:6800/3901761126,v1:172.21.15.5:6801/3901761126] conn(0x7f1f4806f910 msgr2=0x7f1f48071d30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.629+0000 7f1f6b2bf640 1 --2- 172.21.15.5:0/1994964951 >> [v2:172.21.15.5:6800/3901761126,v1:172.21.15.5:6801/3901761126] conn(0x7f1f4806f910 0x7f1f48071d30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.629+0000 7f6e05f1e640 1 --2- 172.21.15.5:0/2182030469 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6de4067660 0x7f6de4069b20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6df80ae000 tx=0x7f6df80a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.629+0000 7f1f6b2bf640 1 -- 172.21.15.5:0/1994964951 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f1f48067690 msgr2=0x7f1f48069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.629+0000 7f1f6b2bf640 1 --2- 172.21.15.5:0/1994964951 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f1f48067690 0x7f1f48069b50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0012f0 tx=0x7f1f5c0017b0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.629+0000 7f1f6b2bf640 1 -- 172.21.15.5:0/1994964951 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f640b1540 msgr2=0x7f1f641e24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.629+0000 7f1f6b2bf640 1 --2- 172.21.15.5:0/1994964951 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f640b1540 0x7f1f641e24d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1f60019dc0 tx=0x7f1f60019040 comp rx=0 tx=0).stop 2024-03-18T15:33:48.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.629+0000 7f1f6985c640 1 -- 172.21.15.5:0/1994964951 reap_dead start 2024-03-18T15:33:48.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.629+0000 7f1f6b2bf640 1 -- 172.21.15.5:0/1994964951 shutdown_connections 2024-03-18T15:33:48.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.629+0000 7f1f6b2bf640 1 -- 172.21.15.5:0/1994964951 >> 172.21.15.5:0/1994964951 conn(0x7f1f64066fc0 msgr2=0x7f1f640b8210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:48.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.629+0000 7f4297fff640 1 Processor -- start 2024-03-18T15:33:48.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.629+0000 7f1f6b2bf640 1 -- 172.21.15.5:0/1994964951 shutdown_connections 2024-03-18T15:33:48.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.629+0000 7f4297fff640 1 -- start start 2024-03-18T15:33:48.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.633+0000 7f4297fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42900c5ea0 0x7f4290168540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.633+0000 7f4297fff640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f42900cc3c0 0x7f42900d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.633+0000 7f4297fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42900d1480 0x7f4290021e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.633+0000 7f4297fff640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f42900c9b40 con 0x7f42900c5ea0 2024-03-18T15:33:48.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.633+0000 7f4297fff640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f42900c99c0 con 0x7f42900d1480 2024-03-18T15:33:48.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.633+0000 7f4297fff640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f42900cce80 con 0x7f42900cc3c0 2024-03-18T15:33:48.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.633+0000 7f6df67fc640 1 -- 172.21.15.5:0/2182030469 <== osd.7 v2:172.21.15.80:6800/982831666 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6de4072350 con 0x7f6de406f880 2024-03-18T15:33:48.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.633+0000 7f4296ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42900c5ea0 0x7f4290168540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.633+0000 7f4296ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42900c5ea0 0x7f4290168540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38516/0 (socket says 172.21.15.5:38516) 2024-03-18T15:33:48.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.633+0000 7f4296ffd640 1 -- 172.21.15.5:0/1694852122 learned_addr learned my addr 172.21.15.5:0/1694852122 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:48.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.633+0000 7f42977fe640 1 --2- 172.21.15.5:0/1694852122 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42900d1480 0x7f4290021e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.633+0000 7f4296ffd640 1 -- 172.21.15.5:0/1694852122 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f42900cc3c0 msgr2=0x7f42900d2ed0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-18T15:33:48.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.633+0000 7f4296ffd640 1 --2- 172.21.15.5:0/1694852122 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f42900cc3c0 0x7f42900d2ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.633+0000 7f4296ffd640 1 -- 172.21.15.5:0/1694852122 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42900d1480 msgr2=0x7f4290021e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.633+0000 7f4296ffd640 1 --2- 172.21.15.5:0/1694852122 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42900d1480 0x7f4290021e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.633+0000 7f4296ffd640 1 -- 172.21.15.5:0/1694852122 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4290023a70 con 0x7f42900c5ea0 2024-03-18T15:33:48.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.633+0000 7f4296ffd640 1 --2- 172.21.15.5:0/1694852122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42900c5ea0 0x7f4290168540 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f428800c530 tx=0x7f428800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.633+0000 7f090e2e1640 1 --2- 172.21.15.5:0/3535181337 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09000c47c0 0x7f0900039770 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0908000b70 tx=0x7f0908097bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c584d8d3135e0116 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.633+0000 7f427f7fe640 1 -- 172.21.15.5:0/1694852122 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4288016020 con 0x7f42900c5ea0 2024-03-18T15:33:48.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.633+0000 7f427f7fe640 1 -- 172.21.15.5:0/1694852122 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4288005220 con 0x7f42900c5ea0 2024-03-18T15:33:48.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.633+0000 7f427f7fe640 1 -- 172.21.15.5:0/1694852122 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4288007030 con 0x7f42900c5ea0 2024-03-18T15:33:48.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.633+0000 7f090cade640 1 -- 172.21.15.5:0/3535181337 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0908128070 con 0x7f09000c47c0 2024-03-18T15:33:48.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.633+0000 7f090cade640 1 -- 172.21.15.5:0/3535181337 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0908124670 con 0x7f09000c47c0 2024-03-18T15:33:48.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.633+0000 7f090cade640 1 -- 172.21.15.5:0/3535181337 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0908124aa0 con 0x7f09000c47c0 2024-03-18T15:33:48.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.633+0000 7f4297fff640 1 -- 172.21.15.5:0/1694852122 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42900232b0 con 0x7f42900c5ea0 2024-03-18T15:33:48.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.633+0000 7f4297fff640 1 -- 172.21.15.5:0/1694852122 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4290034f80 con 0x7f42900c5ea0 2024-03-18T15:33:48.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.633+0000 7f4297fff640 1 -- 172.21.15.5:0/1694852122 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4290005720 con 0x7f42900c5ea0 2024-03-18T15:33:48.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.633+0000 7f090eae2640 1 -- 172.21.15.5:0/3535181337 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09000c47c0 msgr2=0x7f0900039770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.633+0000 7f090eae2640 1 --2- 172.21.15.5:0/3535181337 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09000c47c0 0x7f0900039770 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0908000b70 tx=0x7f0908097bd0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.633+0000 7f090eae2640 1 -- 172.21.15.5:0/3535181337 shutdown_connections 2024-03-18T15:33:48.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.633+0000 7f090eae2640 1 --2- 172.21.15.5:0/3535181337 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09000c47c0 0x7f0900039770 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.633+0000 7f090eae2640 1 -- 172.21.15.5:0/3535181337 >> 172.21.15.5:0/3535181337 conn(0x7f090003c6b0 msgr2=0x7f090003cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:48.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.633+0000 7f427f7fe640 1 -- 172.21.15.5:0/1694852122 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f42880071d0 con 0x7f42900c5ea0 2024-03-18T15:33:48.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.633+0000 7f427f7fe640 1 --2- 172.21.15.5:0/1694852122 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f4284067660 0x7f4284069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.633+0000 7f427f7fe640 1 -- 172.21.15.5:0/1694852122 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f428808a050 con 0x7f42900c5ea0 2024-03-18T15:33:48.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.633+0000 7f42967fc640 1 --2- 172.21.15.5:0/1694852122 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f4284067660 0x7f4284069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.633+0000 7f427f7fe640 1 --2- 172.21.15.5:0/1694852122 >> [v2:172.21.15.5:6802/976264754,v1:172.21.15.5:6804/976264754] conn(0x7f428406f860 0x7f4284071c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.633+0000 7f427f7fe640 1 -- 172.21.15.5:0/1694852122 --> [v2:172.21.15.5:6802/976264754,v1:172.21.15.5:6804/976264754] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4288020e50 con 0x7f428406f860 2024-03-18T15:33:48.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.633+0000 7f427f7fe640 1 -- 172.21.15.5:0/1694852122 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f4288085450 con 0x7f42900c5ea0 2024-03-18T15:33:48.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.637+0000 7f42977fe640 1 --2- 172.21.15.5:0/1694852122 >> [v2:172.21.15.5:6802/976264754,v1:172.21.15.5:6804/976264754] conn(0x7f428406f860 0x7f4284071c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.637+0000 7f42977fe640 1 --2- 172.21.15.5:0/1694852122 >> [v2:172.21.15.5:6802/976264754,v1:172.21.15.5:6804/976264754] conn(0x7f428406f860 0x7f4284071c80 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.637+0000 7f42967fc640 1 --2- 172.21.15.5:0/1694852122 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f4284067660 0x7f4284069b20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f42900ca800 tx=0x7f428c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.637+0000 7f427f7fe640 1 -- 172.21.15.5:0/1694852122 <== osd.0 v2:172.21.15.5:6802/976264754 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4288020e50 con 0x7f428406f860 2024-03-18T15:33:48.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.637+0000 7fe55aebf640 1 -- 172.21.15.5:0/2811507121 --> [v2:172.21.15.80:6824/627153447,v1:172.21.15.80:6825/627153447] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe534002de0 con 0x7fe53806f950 2024-03-18T15:33:48.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.637+0000 7f1f6b2bf640 1 -- 172.21.15.5:0/1994964951 wait complete. 2024-03-18T15:33:48.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.637+0000 7fe5517fa640 1 -- 172.21.15.5:0/2811507121 <== osd.5 v2:172.21.15.80:6824/627153447 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe534002de0 con 0x7fe53806f950 2024-03-18T15:33:48.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.637+0000 7fe55aebf640 1 -- 172.21.15.5:0/2811507121 >> [v2:172.21.15.80:6824/627153447,v1:172.21.15.80:6825/627153447] conn(0x7fe53806f950 msgr2=0x7fe538071d70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.637+0000 7fe55aebf640 1 --2- 172.21.15.5:0/2811507121 >> [v2:172.21.15.80:6824/627153447,v1:172.21.15.80:6825/627153447] conn(0x7fe53806f950 0x7fe538071d70 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.637+0000 7fe55aebf640 1 -- 172.21.15.5:0/2811507121 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe538067730 msgr2=0x7fe538069bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.637+0000 7fe55aebf640 1 --2- 172.21.15.5:0/2811507121 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe538067730 0x7fe538069bf0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe5480af000 tx=0x7fe5480b7040 comp rx=0 tx=0).stop 2024-03-18T15:33:48.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.637+0000 7fe55aebf640 1 -- 172.21.15.5:0/2811507121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe55413f460 msgr2=0x7fe554140d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.637+0000 7fe55aebf640 1 --2- 172.21.15.5:0/2811507121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe55413f460 0x7fe554140d40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe54c007d40 tx=0x7fe54c00f6b0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.637+0000 7fe55945c640 1 -- 172.21.15.5:0/2811507121 reap_dead start 2024-03-18T15:33:48.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.637+0000 7fe55aebf640 1 -- 172.21.15.5:0/2811507121 shutdown_connections 2024-03-18T15:33:48.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.637+0000 7fe55aebf640 1 -- 172.21.15.5:0/2811507121 >> 172.21.15.5:0/2811507121 conn(0x7fe5540b7a80 msgr2=0x7fe554142dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:48.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.637+0000 7fe55aebf640 1 -- 172.21.15.5:0/2811507121 shutdown_connections 2024-03-18T15:33:48.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.641+0000 7fe55aebf640 1 -- 172.21.15.5:0/2811507121 wait complete. 2024-03-18T15:33:48.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.641+0000 7f090eae2640 1 -- 172.21.15.5:0/3535181337 shutdown_connections 2024-03-18T15:33:48.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.641+0000 7f090eae2640 1 -- 172.21.15.5:0/3535181337 wait complete. 2024-03-18T15:33:48.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.641+0000 7fc20f2ce640 1 -- 172.21.15.5:0/1842036320 --> [v2:172.21.15.5:6824/2425932735,v1:172.21.15.5:6825/2425932735] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc20804f550 con 0x7fc1ec06f8d0 2024-03-18T15:33:48.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.641+0000 7fc1fdffb640 1 -- 172.21.15.5:0/1842036320 <== osd.3 v2:172.21.15.5:6824/2425932735 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc20804f550 con 0x7fc1ec06f8d0 2024-03-18T15:33:48.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.645+0000 7fc20f2ce640 1 -- 172.21.15.5:0/1842036320 >> [v2:172.21.15.5:6824/2425932735,v1:172.21.15.5:6825/2425932735] conn(0x7fc1ec06f8d0 msgr2=0x7fc1ec071cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.645+0000 7fc20f2ce640 1 --2- 172.21.15.5:0/1842036320 >> [v2:172.21.15.5:6824/2425932735,v1:172.21.15.5:6825/2425932735] conn(0x7fc1ec06f8d0 0x7fc1ec071cf0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.645+0000 7f090eae2640 1 Processor -- start 2024-03-18T15:33:48.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.645+0000 7fc20f2ce640 1 -- 172.21.15.5:0/1842036320 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fc1ec0676b0 msgr2=0x7fc1ec069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.645+0000 7fc20f2ce640 1 --2- 172.21.15.5:0/1842036320 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fc1ec0676b0 0x7fc1ec069b70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc204001040 tx=0x7fc204009040 comp rx=0 tx=0).stop 2024-03-18T15:33:48.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.645+0000 7fc20f2ce640 1 -- 172.21.15.5:0/1842036320 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc208072d50 msgr2=0x7fc208144370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.645+0000 7fc20f2ce640 1 --2- 172.21.15.5:0/1842036320 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc208072d50 0x7fc208144370 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc1f80af040 tx=0x7fc1f801abc0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.645+0000 7fc20d86b640 1 -- 172.21.15.5:0/1842036320 reap_dead start 2024-03-18T15:33:48.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.645+0000 7f090eae2640 1 -- start start 2024-03-18T15:33:48.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.645+0000 7fc20f2ce640 1 -- 172.21.15.5:0/1842036320 shutdown_connections 2024-03-18T15:33:48.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.645+0000 7fc20f2ce640 1 -- 172.21.15.5:0/1842036320 >> 172.21.15.5:0/1842036320 conn(0x7fc20806e1c0 msgr2=0x7fc2080b6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:48.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.645+0000 7fc20f2ce640 1 -- 172.21.15.5:0/1842036320 shutdown_connections 2024-03-18T15:33:48.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.645+0000 7fc20f2ce640 1 -- 172.21.15.5:0/1842036320 wait complete. 2024-03-18T15:33:48.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.645+0000 7f090eae2640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0900037340 0x7f0900034de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.645+0000 7f090eae2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0900038e10 0x7f0900035f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.645+0000 7f090eae2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09000c47c0 0x7f0900168fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.645+0000 7f090eae2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f09000085b0 con 0x7f09000c47c0 2024-03-18T15:33:48.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.645+0000 7f090eae2640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0900008430 con 0x7f0900038e10 2024-03-18T15:33:48.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.645+0000 7f090eae2640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f0900008730 con 0x7f0900037340 2024-03-18T15:33:48.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.645+0000 7fa67afe1640 1 -- 172.21.15.5:0/3795426679 --> [v2:172.21.15.80:6803/488342527,v1:172.21.15.80:6805/488342527] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa67413d6c0 con 0x7fa65806f9e0 2024-03-18T15:33:48.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.645+0000 7f090e2e1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09000c47c0 0x7f0900168fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.645+0000 7f090e2e1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09000c47c0 0x7f0900168fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38518/0 (socket says 172.21.15.5:38518) 2024-03-18T15:33:48.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.645+0000 7f090e2e1640 1 -- 172.21.15.5:0/2398547601 learned_addr learned my addr 172.21.15.5:0/2398547601 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:48.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.645+0000 7f090dae0640 1 --2- 172.21.15.5:0/2398547601 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0900037340 0x7f0900034de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.645+0000 7f090d2df640 1 --2- 172.21.15.5:0/2398547601 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0900038e10 0x7f0900035f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.645+0000 7fa6717fa640 1 -- 172.21.15.5:0/3795426679 <== osd.6 v2:172.21.15.80:6803/488342527 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa67413d6c0 con 0x7fa65806f9e0 2024-03-18T15:33:48.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.645+0000 7f090d2df640 1 -- 172.21.15.5:0/2398547601 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0900037340 msgr2=0x7f0900034de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.645+0000 7f090d2df640 1 --2- 172.21.15.5:0/2398547601 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0900037340 0x7f0900034de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.645+0000 7f090d2df640 1 -- 172.21.15.5:0/2398547601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09000c47c0 msgr2=0x7f0900168fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.645+0000 7f090d2df640 1 --2- 172.21.15.5:0/2398547601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09000c47c0 0x7f0900168fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.645+0000 7f090d2df640 1 -- 172.21.15.5:0/2398547601 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09000364d0 con 0x7f0900038e10 2024-03-18T15:33:48.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.645+0000 7f090d2df640 1 --2- 172.21.15.5:0/2398547601 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0900038e10 0x7f0900035f90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0900039220 tx=0x7f090400c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.645+0000 7fa67afe1640 1 -- 172.21.15.5:0/3795426679 >> [v2:172.21.15.80:6803/488342527,v1:172.21.15.80:6805/488342527] conn(0x7fa65806f9e0 msgr2=0x7fa658071e00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.645+0000 7fa67afe1640 1 --2- 172.21.15.5:0/3795426679 >> [v2:172.21.15.80:6803/488342527,v1:172.21.15.80:6805/488342527] conn(0x7fa65806f9e0 0x7fa658071e00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.645+0000 7fa67afe1640 1 -- 172.21.15.5:0/3795426679 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa658067760 msgr2=0x7fa658069c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.645+0000 7fa67afe1640 1 --2- 172.21.15.5:0/3795426679 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa658067760 0x7fa658069c20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa6680ad000 tx=0x7fa6680a6000 comp rx=0 tx=0).stop 2024-03-18T15:33:48.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.645+0000 7fa67afe1640 1 -- 172.21.15.5:0/3795426679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa674071240 msgr2=0x7fa6741e7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.645+0000 7fa67afe1640 1 --2- 172.21.15.5:0/3795426679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa674071240 0x7fa6741e7cb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa66000e930 tx=0x7fa66000edf0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.645+0000 7fa67957e640 1 -- 172.21.15.5:0/3795426679 reap_dead start 2024-03-18T15:33:48.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.645+0000 7fa67afe1640 1 -- 172.21.15.5:0/3795426679 shutdown_connections 2024-03-18T15:33:48.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.649+0000 7fa67afe1640 1 -- 172.21.15.5:0/3795426679 >> 172.21.15.5:0/3795426679 conn(0x7fa6740b7a80 msgr2=0x7fa67406bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:48.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.649+0000 7fa67afe1640 1 -- 172.21.15.5:0/3795426679 shutdown_connections 2024-03-18T15:33:48.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.649+0000 7fa67afe1640 1 -- 172.21.15.5:0/3795426679 wait complete. 2024-03-18T15:33:48.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.653+0000 7f08fa7fc640 1 -- 172.21.15.5:0/2398547601 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0904017070 con 0x7f0900038e10 2024-03-18T15:33:48.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.653+0000 7f08fa7fc640 1 -- 172.21.15.5:0/2398547601 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0904003600 con 0x7f0900038e10 2024-03-18T15:33:48.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.653+0000 7f090eae2640 1 -- 172.21.15.5:0/2398547601 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0900034940 con 0x7f0900038e10 2024-03-18T15:33:48.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.653+0000 7f090eae2640 1 -- 172.21.15.5:0/2398547601 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0900033cd0 con 0x7f0900038e10 2024-03-18T15:33:48.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.653+0000 7f08fa7fc640 1 -- 172.21.15.5:0/2398547601 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09040039f0 con 0x7f0900038e10 2024-03-18T15:33:48.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.653+0000 7f08fa7fc640 1 -- 172.21.15.5:0/2398547601 <== mon.1 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f090401a070 con 0x7f0900038e10 2024-03-18T15:33:48.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.653+0000 7f08fa7fc640 1 --2- 172.21.15.5:0/2398547601 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f08ec0676e0 0x7f08ec069ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.657+0000 7f090dae0640 1 --2- 172.21.15.5:0/2398547601 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f08ec0676e0 0x7f08ec069ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.657+0000 7f08fa7fc640 1 -- 172.21.15.5:0/2398547601 <== mon.1 v2:172.21.15.80:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0904084990 con 0x7f0900038e10 2024-03-18T15:33:48.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.657+0000 7f090dae0640 1 --2- 172.21.15.5:0/2398547601 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f08ec0676e0 0x7f08ec069ba0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f08fc001040 tx=0x7f08fc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.678 INFO:teuthology.orchestra.run.smithi005.stdout:60129542150 2024-03-18T15:33:48.678 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-18T15:33:48.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.657+0000 7f6e08182640 1 -- 172.21.15.5:0/2182030469 --> [v2:172.21.15.80:6800/982831666,v1:172.21.15.80:6801/982831666] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6e00151ad0 con 0x7f6de406f880 2024-03-18T15:33:48.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.661+0000 7f090eae2640 1 --2- 172.21.15.5:0/2398547601 >> [v2:172.21.15.80:6816/976514285,v1:172.21.15.80:6817/976514285] conn(0x7f0900033f10 0x7f09000031a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.661+0000 7f090eae2640 1 -- 172.21.15.5:0/2398547601 --> [v2:172.21.15.80:6816/976514285,v1:172.21.15.80:6817/976514285] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f09000dbe80 con 0x7f0900033f10 2024-03-18T15:33:48.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.661+0000 7f6df67fc640 1 -- 172.21.15.5:0/2182030469 <== osd.7 v2:172.21.15.80:6800/982831666 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6e00151ad0 con 0x7f6de406f880 2024-03-18T15:33:48.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.661+0000 7f4297fff640 1 -- 172.21.15.5:0/1694852122 --> [v2:172.21.15.5:6802/976264754,v1:172.21.15.5:6804/976264754] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f429000baf0 con 0x7f428406f860 2024-03-18T15:33:48.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.661+0000 7f090e2e1640 1 --2- 172.21.15.5:0/2398547601 >> [v2:172.21.15.80:6816/976514285,v1:172.21.15.80:6817/976514285] conn(0x7f0900033f10 0x7f09000031a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.661+0000 7f427f7fe640 1 -- 172.21.15.5:0/1694852122 <== osd.0 v2:172.21.15.5:6802/976264754 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f429000baf0 con 0x7f428406f860 2024-03-18T15:33:48.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.661+0000 7f4297fff640 1 -- 172.21.15.5:0/1694852122 >> [v2:172.21.15.5:6802/976264754,v1:172.21.15.5:6804/976264754] conn(0x7f428406f860 msgr2=0x7f4284071c80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.661+0000 7f4297fff640 1 --2- 172.21.15.5:0/1694852122 >> [v2:172.21.15.5:6802/976264754,v1:172.21.15.5:6804/976264754] conn(0x7f428406f860 0x7f4284071c80 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.661+0000 7f090e2e1640 1 --2- 172.21.15.5:0/2398547601 >> [v2:172.21.15.80:6816/976514285,v1:172.21.15.80:6817/976514285] conn(0x7f0900033f10 0x7f09000031a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.661+0000 7f4297fff640 1 -- 172.21.15.5:0/1694852122 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f4284067660 msgr2=0x7f4284069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.661+0000 7f4297fff640 1 --2- 172.21.15.5:0/1694852122 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f4284067660 0x7f4284069b20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f42900ca800 tx=0x7f428c008040 comp rx=0 tx=0).stop 2024-03-18T15:33:48.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.661+0000 7f6e08182640 1 -- 172.21.15.5:0/2182030469 >> [v2:172.21.15.80:6800/982831666,v1:172.21.15.80:6801/982831666] conn(0x7f6de406f880 msgr2=0x7f6de4071ca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.661+0000 7f6e08182640 1 --2- 172.21.15.5:0/2182030469 >> [v2:172.21.15.80:6800/982831666,v1:172.21.15.80:6801/982831666] conn(0x7f6de406f880 0x7f6de4071ca0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.661+0000 7f4297fff640 1 -- 172.21.15.5:0/1694852122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42900c5ea0 msgr2=0x7f4290168540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.661+0000 7f4297fff640 1 --2- 172.21.15.5:0/1694852122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42900c5ea0 0x7f4290168540 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f428800c530 tx=0x7f428800c9f0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.661+0000 7f42977fe640 1 -- 172.21.15.5:0/1694852122 reap_dead start 2024-03-18T15:33:48.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.661+0000 7f4297fff640 1 -- 172.21.15.5:0/1694852122 shutdown_connections 2024-03-18T15:33:48.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.661+0000 7f4297fff640 1 -- 172.21.15.5:0/1694852122 >> 172.21.15.5:0/1694852122 conn(0x7f429000ab50 msgr2=0x7f4290039360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:48.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.661+0000 7f4297fff640 1 -- 172.21.15.5:0/1694852122 shutdown_connections 2024-03-18T15:33:48.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.661+0000 7f6e08182640 1 -- 172.21.15.5:0/2182030469 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6de4067660 msgr2=0x7f6de4069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.661+0000 7f6e08182640 1 --2- 172.21.15.5:0/2182030469 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6de4067660 0x7f6de4069b20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6df80ae000 tx=0x7f6df80a7070 comp rx=0 tx=0).stop 2024-03-18T15:33:48.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.661+0000 7f6e08182640 1 -- 172.21.15.5:0/2182030469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e000b3440 msgr2=0x7f6e0013ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.661+0000 7f4297fff640 1 -- 172.21.15.5:0/1694852122 wait complete. 2024-03-18T15:33:48.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.661+0000 7f6e08182640 1 --2- 172.21.15.5:0/2182030469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e000b3440 0x7f6e0013ca00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc00e7f0 tx=0x7f6dfc00ecb0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.661+0000 7f6e0671f640 1 -- 172.21.15.5:0/2182030469 reap_dead start 2024-03-18T15:33:48.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.661+0000 7f6e08182640 1 -- 172.21.15.5:0/2182030469 shutdown_connections 2024-03-18T15:33:48.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.661+0000 7f6e08182640 1 -- 172.21.15.5:0/2182030469 >> 172.21.15.5:0/2182030469 conn(0x7f6e000b7a80 msgr2=0x7f6e0006d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:48.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.661+0000 7f6e08182640 1 -- 172.21.15.5:0/2182030469 shutdown_connections 2024-03-18T15:33:48.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.661+0000 7f6e08182640 1 -- 172.21.15.5:0/2182030469 wait complete. 2024-03-18T15:33:48.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.661+0000 7f08fa7fc640 1 -- 172.21.15.5:0/2398547601 <== osd.4 v2:172.21.15.80:6816/976514285 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f09000dbe80 con 0x7f0900033f10 2024-03-18T15:33:48.685 INFO:teuthology.orchestra.run.smithi005.stdout:60129542150 2024-03-18T15:33:48.685 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-18T15:33:48.686 INFO:teuthology.orchestra.run.smithi005.stdout:60129542150 2024-03-18T15:33:48.687 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-18T15:33:48.687 INFO:teuthology.orchestra.run.smithi005.stdout:64424509447 2024-03-18T15:33:48.688 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-18T15:33:48.688 INFO:teuthology.orchestra.run.smithi005.stdout:64424509447 2024-03-18T15:33:48.689 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-18T15:33:48.690 INFO:teuthology.orchestra.run.smithi005.stdout:60129542151 2024-03-18T15:33:48.690 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-18T15:33:48.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.689+0000 7f090eae2640 1 -- 172.21.15.5:0/2398547601 --> [v2:172.21.15.80:6816/976514285,v1:172.21.15.80:6817/976514285] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f09000c8d40 con 0x7f0900033f10 2024-03-18T15:33:48.695 INFO:teuthology.orchestra.run.smithi005.stdout:60129542149 2024-03-18T15:33:48.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.689+0000 7f08fa7fc640 1 -- 172.21.15.5:0/2398547601 <== osd.4 v2:172.21.15.80:6816/976514285 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f09000c8d40 con 0x7f0900033f10 2024-03-18T15:33:48.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.689+0000 7f090eae2640 1 -- 172.21.15.5:0/2398547601 >> [v2:172.21.15.80:6816/976514285,v1:172.21.15.80:6817/976514285] conn(0x7f0900033f10 msgr2=0x7f09000031a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.689+0000 7f090eae2640 1 --2- 172.21.15.5:0/2398547601 >> [v2:172.21.15.80:6816/976514285,v1:172.21.15.80:6817/976514285] conn(0x7f0900033f10 0x7f09000031a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.693+0000 7f090eae2640 1 -- 172.21.15.5:0/2398547601 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f08ec0676e0 msgr2=0x7f08ec069ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.693+0000 7f090eae2640 1 --2- 172.21.15.5:0/2398547601 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f08ec0676e0 0x7f08ec069ba0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f08fc001040 tx=0x7f08fc009040 comp rx=0 tx=0).stop 2024-03-18T15:33:48.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.693+0000 7f090eae2640 1 -- 172.21.15.5:0/2398547601 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0900038e10 msgr2=0x7f0900035f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.693+0000 7f090eae2640 1 --2- 172.21.15.5:0/2398547601 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0900038e10 0x7f0900035f90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0900039220 tx=0x7f090400c870 comp rx=0 tx=0).stop 2024-03-18T15:33:48.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.693+0000 7f090e2e1640 1 -- 172.21.15.5:0/2398547601 reap_dead start 2024-03-18T15:33:48.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.693+0000 7f090eae2640 1 -- 172.21.15.5:0/2398547601 shutdown_connections 2024-03-18T15:33:48.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.693+0000 7f090eae2640 1 -- 172.21.15.5:0/2398547601 >> 172.21.15.5:0/2398547601 conn(0x7f090003c6b0 msgr2=0x7f09000c35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:48.697 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-18T15:33:48.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.693+0000 7f090eae2640 1 -- 172.21.15.5:0/2398547601 shutdown_connections 2024-03-18T15:33:48.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.693+0000 7f090eae2640 1 -- 172.21.15.5:0/2398547601 wait complete. 2024-03-18T15:33:48.720 INFO:teuthology.orchestra.run.smithi005.stdout:60129542152 2024-03-18T15:33:48.721 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-18T15:33:48.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.829+0000 7f887910a640 1 Processor -- start 2024-03-18T15:33:48.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.829+0000 7f887910a640 1 -- start start 2024-03-18T15:33:48.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.829+0000 7f887910a640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f88740b1540 0x7f887414ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.829+0000 7f887910a640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88741477f0 con 0x7f88740b1160 2024-03-18T15:33:48.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.829+0000 7f887910a640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88740afe60 con 0x7f88740b4ee0 2024-03-18T15:33:48.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.829+0000 7f887910a640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f88740b0140 con 0x7f88740b1540 2024-03-18T15:33:48.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.829+0000 7f8872d9d640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f88740b1160 0x7f88740b5ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:57256/0 (socket says 172.21.15.5:57256) 2024-03-18T15:33:48.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.829+0000 7f8872d9d640 1 -- 172.21.15.5:0/3504952017 learned_addr learned my addr 172.21.15.5:0/3504952017 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:33:48.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.829+0000 7f887359e640 1 --2- 172.21.15.5:0/3504952017 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f88740b1540 0x7f887414ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.829+0000 7f8871d9b640 1 -- 172.21.15.5:0/3504952017 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2418633122 0 0) 0x7f88741477f0 con 0x7f88740b1160 2024-03-18T15:33:48.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.829+0000 7f8871d9b640 1 -- 172.21.15.5:0/3504952017 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f885c003530 con 0x7f88740b1160 2024-03-18T15:33:48.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.829+0000 7f8871d9b640 1 -- 172.21.15.5:0/3504952017 <== mon.1 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 183780418 0 0) 0x7f88740afe60 con 0x7f88740b4ee0 2024-03-18T15:33:48.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.829+0000 7f8871d9b640 1 -- 172.21.15.5:0/3504952017 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f88741477f0 con 0x7f88740b4ee0 2024-03-18T15:33:48.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.829+0000 7f887359e640 1 -- 172.21.15.5:0/3504952017 >> v1:172.21.15.80:6789/0 conn(0x7f88740b4ee0 legacy=0x7f88741470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.829+0000 7f887359e640 1 -- 172.21.15.5:0/3504952017 >> v1:172.21.15.5:6789/0 conn(0x7f88740b1160 legacy=0x7f88740b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.829+0000 7f887359e640 1 -- 172.21.15.5:0/3504952017 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88740b0ab0 con 0x7f88740b1540 2024-03-18T15:33:48.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.833+0000 7f887359e640 1 --2- 172.21.15.5:0/3504952017 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f88740b1540 0x7f887414ec30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f886800a990 tx=0x7f886801e880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72aa3c45420eba74 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.833+0000 7f8871d9b640 1 -- 172.21.15.5:0/3504952017 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88680b4070 con 0x7f88740b1540 2024-03-18T15:33:48.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.833+0000 7f8871d9b640 1 -- 172.21.15.5:0/3504952017 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88680a7820 con 0x7f88740b1540 2024-03-18T15:33:48.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.833+0000 7f8871d9b640 1 -- 172.21.15.5:0/3504952017 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f886800b040 con 0x7f88740b1540 2024-03-18T15:33:48.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.837+0000 7f887910a640 1 -- 172.21.15.5:0/3504952017 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f88740b1540 msgr2=0x7f887414ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.837+0000 7f887910a640 1 --2- 172.21.15.5:0/3504952017 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f88740b1540 0x7f887414ec30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f886800a990 tx=0x7f886801e880 comp rx=0 tx=0).stop 2024-03-18T15:33:48.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.837+0000 7f887910a640 1 -- 172.21.15.5:0/3504952017 shutdown_connections 2024-03-18T15:33:48.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.837+0000 7f887910a640 1 --2- 172.21.15.5:0/3504952017 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f88740b1540 0x7f887414ec30 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.837+0000 7f887910a640 1 -- 172.21.15.5:0/3504952017 >> 172.21.15.5:0/3504952017 conn(0x7f8874066fc0 msgr2=0x7f88740673d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:48.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.837+0000 7f887910a640 1 -- 172.21.15.5:0/3504952017 shutdown_connections 2024-03-18T15:33:48.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.837+0000 7f887910a640 1 -- 172.21.15.5:0/3504952017 wait complete. 2024-03-18T15:33:48.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.837+0000 7f887910a640 1 Processor -- start 2024-03-18T15:33:48.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.837+0000 7f887910a640 1 -- start start 2024-03-18T15:33:48.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.837+0000 7f887910a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88740b1160 0x7f88741e7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.837+0000 7f887910a640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f88740b1540 0x7f8874136db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.837+0000 7f887910a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f88740b4ee0 0x7f88740718b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.837+0000 7f887910a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8874146a70 con 0x7f88740b1160 2024-03-18T15:33:48.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.837+0000 7f887910a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f88741468f0 con 0x7f88740b4ee0 2024-03-18T15:33:48.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.837+0000 7f887910a640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f8874151990 con 0x7f88740b1540 2024-03-18T15:33:48.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.837+0000 7f8872d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88740b1160 0x7f88741e7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.837+0000 7f8872d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88740b1160 0x7f88741e7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38520/0 (socket says 172.21.15.5:38520) 2024-03-18T15:33:48.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.837+0000 7f8872d9d640 1 -- 172.21.15.5:0/2278470046 learned_addr learned my addr 172.21.15.5:0/2278470046 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:48.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.837+0000 7f887259c640 1 --2- 172.21.15.5:0/2278470046 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f88740b1540 0x7f8874136db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.837+0000 7f887359e640 1 --2- 172.21.15.5:0/2278470046 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f88740b4ee0 0x7f88740718b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.841+0000 7f8872d9d640 1 -- 172.21.15.5:0/2278470046 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f88740b1540 msgr2=0x7f8874136db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.841+0000 7f8872d9d640 1 --2- 172.21.15.5:0/2278470046 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f88740b1540 0x7f8874136db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.841+0000 7f8872d9d640 1 -- 172.21.15.5:0/2278470046 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f88740b4ee0 msgr2=0x7f88740718b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.841+0000 7f8872d9d640 1 --2- 172.21.15.5:0/2278470046 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f88740b4ee0 0x7f88740718b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.841+0000 7f8872d9d640 1 -- 172.21.15.5:0/2278470046 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f887406fc00 con 0x7f88740b1160 2024-03-18T15:33:48.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.841+0000 7f887359e640 1 --2- 172.21.15.5:0/2278470046 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f88740b4ee0 0x7f88740718b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:33:48.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.841+0000 7f8872d9d640 1 --2- 172.21.15.5:0/2278470046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88740b1160 0x7f88741e7c00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f886400d690 tx=0x7f886400db50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.841+0000 7f885b7fe640 1 -- 172.21.15.5:0/2278470046 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8864007050 con 0x7f88740b1160 2024-03-18T15:33:48.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.841+0000 7f0a9e35e640 1 Processor -- start 2024-03-18T15:33:48.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.841+0000 7f0a9e35e640 1 -- start start 2024-03-18T15:33:48.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.841+0000 7f887910a640 1 -- 172.21.15.5:0/2278470046 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f887406fe70 con 0x7f88740b1160 2024-03-18T15:33:48.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.841+0000 7f887910a640 1 -- 172.21.15.5:0/2278470046 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88741e7400 con 0x7f88740b1160 2024-03-18T15:33:48.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.845+0000 7f885b7fe640 1 -- 172.21.15.5:0/2278470046 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8864008970 con 0x7f88740b1160 2024-03-18T15:33:48.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.845+0000 7f0a9e35e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0a980b7850 0x7f0a980729a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.845+0000 7f0a9e35e640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a980b7c30 0x7f0a98145920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.845+0000 7f0a9e35e640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a980b8c30 con 0x7f0a98145250 2024-03-18T15:33:48.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.845+0000 7f0a9e35e640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f0a980b9c20 con 0x7f0a980b7850 2024-03-18T15:33:48.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.845+0000 7f0a9e35e640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f0a980b9f00 con 0x7f0a980b7c30 2024-03-18T15:33:48.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.845+0000 7f0a9c8fb640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a980b7c30 0x7f0a98145920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.845+0000 7f0a9c8fb640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a980b7c30 0x7f0a98145920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:52944/0 (socket says 172.21.15.5:52944) 2024-03-18T15:33:48.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.845+0000 7f0a9c8fb640 1 -- 172.21.15.5:0/3220779581 learned_addr learned my addr 172.21.15.5:0/3220779581 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:48.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.845+0000 7f885b7fe640 1 -- 172.21.15.5:0/2278470046 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8864008da0 con 0x7f88740b1160 2024-03-18T15:33:48.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.845+0000 7f0a9c8fb640 1 -- 172.21.15.5:0/3220779581 >> v2:172.21.15.5:3300/0 conn(0x7f0a980b7850 msgr2=0x7f0a980729a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:48.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.845+0000 7f0a97fff640 1 --2- 172.21.15.5:0/3220779581 >> v2:172.21.15.5:3300/0 conn(0x7f0a980b7850 0x7f0a980729a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.845+0000 7f887910a640 1 -- 172.21.15.5:0/2278470046 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88540052a0 con 0x7f88740b1160 2024-03-18T15:33:48.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.845+0000 7f0a9c8fb640 1 --2- 172.21.15.5:0/3220779581 >> v2:172.21.15.5:3300/0 conn(0x7f0a980b7850 0x7f0a980729a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.845+0000 7f0a9c8fb640 1 -- 172.21.15.5:0/3220779581 >> v1:172.21.15.5:6789/0 conn(0x7f0a98145250 legacy=0x7f0a980b9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.845+0000 7f0a9c8fb640 1 -- 172.21.15.5:0/3220779581 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a980b8f10 con 0x7f0a980b7c30 2024-03-18T15:33:48.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.845+0000 7f0a97fff640 1 --2- 172.21.15.5:0/3220779581 >> v2:172.21.15.5:3300/0 conn(0x7f0a980b7850 0x7f0a980729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:33:48.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.845+0000 7f885b7fe640 1 -- 172.21.15.5:0/2278470046 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f8864010420 con 0x7f88740b1160 2024-03-18T15:33:48.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.849+0000 7f885b7fe640 1 --2- 172.21.15.5:0/2278470046 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f88600676b0 0x7f8860069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.849+0000 7f885b7fe640 1 -- 172.21.15.5:0/2278470046 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8864085ba0 con 0x7f88740b1160 2024-03-18T15:33:48.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.849+0000 7f887259c640 1 --2- 172.21.15.5:0/2278470046 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f88600676b0 0x7f8860069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.849+0000 7f887259c640 1 --2- 172.21.15.5:0/2278470046 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f88600676b0 0x7f8860069b70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f88740af910 tx=0x7f886c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.849+0000 7f0a9c8fb640 1 --2- 172.21.15.5:0/3220779581 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a980b7c30 0x7f0a98145920 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0a90009e50 tx=0x7f0a9001c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53bc7138a26a0052 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.849+0000 7f0a977fe640 1 -- 172.21.15.5:0/3220779581 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a900a56e0 con 0x7f0a980b7c30 2024-03-18T15:33:48.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.849+0000 7f0a977fe640 1 -- 172.21.15.5:0/3220779581 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a900a5880 con 0x7f0a980b7c30 2024-03-18T15:33:48.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.849+0000 7f0a977fe640 1 -- 172.21.15.5:0/3220779581 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a900a5c50 con 0x7f0a980b7c30 2024-03-18T15:33:48.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.849+0000 7f0a9e35e640 1 -- 172.21.15.5:0/3220779581 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a980b7c30 msgr2=0x7f0a98145920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.849+0000 7f0a9e35e640 1 --2- 172.21.15.5:0/3220779581 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a980b7c30 0x7f0a98145920 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0a90009e50 tx=0x7f0a9001c910 comp rx=0 tx=0).stop 2024-03-18T15:33:48.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.849+0000 7f0a9e35e640 1 -- 172.21.15.5:0/3220779581 shutdown_connections 2024-03-18T15:33:48.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.849+0000 7f0a9e35e640 1 --2- 172.21.15.5:0/3220779581 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a980b7c30 0x7f0a98145920 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.849+0000 7f0a9e35e640 1 --2- 172.21.15.5:0/3220779581 >> v2:172.21.15.5:3300/0 conn(0x7f0a980b7850 0x7f0a980729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.849+0000 7f0a9e35e640 1 -- 172.21.15.5:0/3220779581 >> 172.21.15.5:0/3220779581 conn(0x7f0a98067190 msgr2=0x7f0a980675a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:48.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.857+0000 7f0a9e35e640 1 -- 172.21.15.5:0/3220779581 shutdown_connections 2024-03-18T15:33:48.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.857+0000 7f0a9e35e640 1 -- 172.21.15.5:0/3220779581 wait complete. 2024-03-18T15:33:48.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.857+0000 7f885b7fe640 1 -- 172.21.15.5:0/2278470046 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f8864014030 con 0x7f88740b1160 2024-03-18T15:33:48.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.857+0000 7f0a9e35e640 1 Processor -- start 2024-03-18T15:33:48.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.857+0000 7f0a9e35e640 1 -- start start 2024-03-18T15:33:48.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.857+0000 7f0a9e35e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a980b7850 0x7f0a98071430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.857+0000 7f0a9e35e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a980b7c30 0x7f0a981d78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.857+0000 7f0a9e35e640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a98145250 0x7f0a981d7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.857+0000 7f0a9e35e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0a981526a0 con 0x7f0a980b7c30 2024-03-18T15:33:48.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.857+0000 7f0a9e35e640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0a98152520 con 0x7f0a980b7850 2024-03-18T15:33:48.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.857+0000 7f0a9e35e640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f0a98152820 con 0x7f0a98145250 2024-03-18T15:33:48.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.857+0000 7f0a9c8fb640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a98145250 0x7f0a981d7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.857+0000 7f0a97fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a980b7850 0x7f0a98071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.857+0000 7f0a9c8fb640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a98145250 0x7f0a981d7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:52960/0 (socket says 172.21.15.5:52960) 2024-03-18T15:33:48.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.857+0000 7f0a9c8fb640 1 -- 172.21.15.5:0/3766413923 learned_addr learned my addr 172.21.15.5:0/3766413923 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:48.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.857+0000 7f0a8ffff640 1 --2- 172.21.15.5:0/3766413923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a980b7c30 0x7f0a981d78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.861+0000 7f0a9c8fb640 1 -- 172.21.15.5:0/3766413923 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a980b7850 msgr2=0x7f0a98071430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.861+0000 7f0a9c8fb640 1 --2- 172.21.15.5:0/3766413923 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a980b7850 0x7f0a98071430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.861+0000 7f0a9c8fb640 1 -- 172.21.15.5:0/3766413923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a980b7c30 msgr2=0x7f0a981d78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.861+0000 7f0a9c8fb640 1 --2- 172.21.15.5:0/3766413923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a980b7c30 0x7f0a981d78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.861+0000 7f0a9c8fb640 1 -- 172.21.15.5:0/3766413923 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a981da2d0 con 0x7f0a98145250 2024-03-18T15:33:48.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.861+0000 7f0a8ffff640 1 --2- 172.21.15.5:0/3766413923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a980b7c30 0x7f0a981d78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:33:48.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.861+0000 7f0a9c8fb640 1 --2- 172.21.15.5:0/3766413923 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a98145250 0x7f0a981d7e30 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0a9001a540 tx=0x7f0a900196b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.861+0000 7f0a957fa640 1 -- 172.21.15.5:0/3766413923 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a900bc020 con 0x7f0a98145250 2024-03-18T15:33:48.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.861+0000 7f0a957fa640 1 -- 172.21.15.5:0/3766413923 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a900b8880 con 0x7f0a98145250 2024-03-18T15:33:48.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.861+0000 7f0a957fa640 1 -- 172.21.15.5:0/3766413923 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a90009510 con 0x7f0a98145250 2024-03-18T15:33:48.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.861+0000 7f0a9e35e640 1 -- 172.21.15.5:0/3766413923 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a981d8910 con 0x7f0a98145250 2024-03-18T15:33:48.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.865+0000 7f0a9e35e640 1 -- 172.21.15.5:0/3766413923 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a981d9af0 con 0x7f0a98145250 2024-03-18T15:33:48.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.865+0000 7f3a118b0640 1 Processor -- start 2024-03-18T15:33:48.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.865+0000 7f3a118b0640 1 -- start start 2024-03-18T15:33:48.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.865+0000 7fbd8696c640 1 Processor -- start 2024-03-18T15:33:48.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.865+0000 7fbd8696c640 1 -- start start 2024-03-18T15:33:48.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.865+0000 7fdf6447a640 1 Processor -- start 2024-03-18T15:33:48.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.865+0000 7f3a118b0640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a04039080 0x7f3a040c4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.865+0000 7f3a108ae640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f3a04038270 0x7f3a040cae40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.5:44124/0 (socket says 172.21.15.5:44124) 2024-03-18T15:33:48.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.865+0000 7f3a118b0640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3a0403a5b0 0x7f3a040c4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.865+0000 7f3a0bfff640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a04039080 0x7f3a040c4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.865+0000 7f3a0bfff640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a04039080 0x7f3a040c4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:52976/0 (socket says 172.21.15.5:52976) 2024-03-18T15:33:48.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.865+0000 7fbd8696c640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbd800b9230 0x7fbd800b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.865+0000 7fbd8696c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fbd800b43d0 0x7fbd800b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.865+0000 7fbd8696c640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd800b4da0 con 0x7fbd800bd5c0 2024-03-18T15:33:48.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.865+0000 7fbd8696c640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fbd800b53d0 con 0x7fbd800b43d0 2024-03-18T15:33:48.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.865+0000 7fbd8696c640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fbd800bd9a0 con 0x7fbd800b9230 2024-03-18T15:33:48.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.865+0000 7f0a957fa640 1 -- 172.21.15.5:0/3766413923 <== mon.2 v2:172.21.15.80:3301/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f0a900bf020 con 0x7f0a98145250 2024-03-18T15:33:48.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.865+0000 7f0a9e35e640 1 -- 172.21.15.5:0/3766413923 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a9804f550 con 0x7f0a98145250 2024-03-18T15:33:48.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.865+0000 7fbd7f7fe640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbd800b9230 0x7fbd800b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.865+0000 7fbd7ffff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fbd800bd5c0 0x7fbd801413f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:57286/0 (socket says 172.21.15.5:57286) 2024-03-18T15:33:48.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.865+0000 7fbd7ffff640 1 -- 172.21.15.5:0/4170349154 learned_addr learned my addr 172.21.15.5:0/4170349154 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:33:48.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.865+0000 7fbd84f09640 1 --2- 172.21.15.5:0/4170349154 >> v2:172.21.15.80:3300/0 conn(0x7fbd800b43d0 0x7fbd800b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.865+0000 7fdf6447a640 1 -- start start 2024-03-18T15:33:48.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.865+0000 7fdf6447a640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fdf5c14f8b0 0x7fdf5c145580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.865+0000 7fdf6447a640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdf5c0b1ce0 0x7fdf5c145ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.865+0000 7fdf6447a640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf5c0b4410 con 0x7fdf5c0b9f40 2024-03-18T15:33:48.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.865+0000 7fdf6447a640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fdf5c0b3860 con 0x7fdf5c14f8b0 2024-03-18T15:33:48.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.865+0000 7fdf6447a640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fdf5c0b32e0 con 0x7fdf5c0b1ce0 2024-03-18T15:33:48.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.865+0000 7f3a108ae640 1 -- 172.21.15.5:0/698750453 learned_addr learned my addr 172.21.15.5:0/698750453 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:33:48.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.865+0000 7f3a110af640 1 --2- 172.21.15.5:0/698750453 >> v2:172.21.15.80:3300/0 conn(0x7f3a0403a5b0 0x7f3a040c4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.865+0000 7fbd7effd640 1 -- 172.21.15.5:0/4170349154 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1276522088 0 0) 0x7fbd800b4da0 con 0x7fbd800bd5c0 2024-03-18T15:33:48.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.865+0000 7fbd7effd640 1 -- 172.21.15.5:0/4170349154 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd68003530 con 0x7fbd800bd5c0 2024-03-18T15:33:48.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.865+0000 7f3a118b0640 1 -- 172.21.15.5:0/698750453 --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a04038650 con 0x7f3a04038270 2024-03-18T15:33:48.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.865+0000 7fdf62216640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fdf5c0b9f40 0x7fdf5c14f1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.5:44134/0 (socket says 172.21.15.5:44134) 2024-03-18T15:33:48.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.865+0000 7fdf62216640 1 -- 172.21.15.5:0/4163366872 learned_addr learned my addr 172.21.15.5:0/4163366872 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:33:48.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.865+0000 7f3a118b0640 1 -- 172.21.15.5:0/698750453 --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f3a04037a30 con 0x7f3a0403a5b0 2024-03-18T15:33:48.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.865+0000 7f3a118b0640 1 -- 172.21.15.5:0/698750453 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f3a040ca640 con 0x7f3a04039080 2024-03-18T15:33:48.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.865+0000 7f0a957fa640 1 --2- 172.21.15.5:0/3766413923 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0a78067760 0x7f0a78069c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.865+0000 7f0a957fa640 1 -- 172.21.15.5:0/3766413923 <== mon.2 v2:172.21.15.80:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0a90120a40 con 0x7f0a98145250 2024-03-18T15:33:48.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.865+0000 7fdf61a15640 1 --2- 172.21.15.5:0/4163366872 >> v2:172.21.15.5:3300/0 conn(0x7fdf5c14f8b0 0x7fdf5c145580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.865+0000 7fbd7f7fe640 1 -- 172.21.15.5:0/4170349154 >> v2:172.21.15.80:3300/0 conn(0x7fbd800b43d0 msgr2=0x7fbd800b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:48.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.865+0000 7fbd7f7fe640 1 --2- 172.21.15.5:0/4170349154 >> v2:172.21.15.80:3300/0 conn(0x7fbd800b43d0 0x7fbd800b4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.865+0000 7fbd7f7fe640 1 -- 172.21.15.5:0/4170349154 >> v1:172.21.15.5:6789/0 conn(0x7fbd800bd5c0 legacy=0x7fbd801413f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.865+0000 7fbd7f7fe640 1 -- 172.21.15.5:0/4170349154 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd800b5670 con 0x7fbd800b9230 2024-03-18T15:33:48.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.865+0000 7fdf62a17640 1 --2- 172.21.15.5:0/4163366872 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdf5c0b1ce0 0x7fdf5c145ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.869+0000 7f0a97fff640 1 --2- 172.21.15.5:0/3766413923 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0a78067760 0x7f0a78069c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.869+0000 7f3a0b7fe640 1 -- 172.21.15.5:0/698750453 <== mon.1 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 74574434 0 0) 0x7f3a04038650 con 0x7f3a04038270 2024-03-18T15:33:48.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.869+0000 7f3a0b7fe640 1 -- 172.21.15.5:0/698750453 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39fc003530 con 0x7f3a04038270 2024-03-18T15:33:48.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.869+0000 7fdf61a15640 1 -- 172.21.15.5:0/4163366872 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdf5c0b1ce0 msgr2=0x7fdf5c145ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:48.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.869+0000 7fdf61a15640 1 --2- 172.21.15.5:0/4163366872 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdf5c0b1ce0 0x7fdf5c145ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.869+0000 7fdf61a15640 1 -- 172.21.15.5:0/4163366872 >> v1:172.21.15.80:6789/0 conn(0x7fdf5c0b9f40 legacy=0x7fdf5c14f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.869+0000 7fdf61a15640 1 -- 172.21.15.5:0/4163366872 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf5c0ba320 con 0x7fdf5c14f8b0 2024-03-18T15:33:48.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.869+0000 7f3a110af640 1 -- 172.21.15.5:0/698750453 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a04039080 msgr2=0x7f3a040c4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:48.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.869+0000 7f3a110af640 1 --2- 172.21.15.5:0/698750453 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a04039080 0x7f3a040c4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.869+0000 7f3a110af640 1 -- 172.21.15.5:0/698750453 >> v1:172.21.15.80:6789/0 conn(0x7f3a04038270 legacy=0x7f3a040cae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.869+0000 7f3a110af640 1 -- 172.21.15.5:0/698750453 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a040ca920 con 0x7f3a0403a5b0 2024-03-18T15:33:48.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.869+0000 7f0a97fff640 1 --2- 172.21.15.5:0/3766413923 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0a78067760 0x7f0a78069c20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0a80006fd0 tx=0x7f0a80008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.869+0000 7f5118467640 1 Processor -- start 2024-03-18T15:33:48.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.869+0000 7f5118467640 1 -- start start 2024-03-18T15:33:48.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.869+0000 7fbd7f7fe640 1 --2- 172.21.15.5:0/4170349154 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbd800b9230 0x7fbd800b31c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbd78009d90 tx=0x7fbd7801c870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e32c6bb3e5f7b78b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.869+0000 7f3a110af640 1 --2- 172.21.15.5:0/698750453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3a0403a5b0 0x7f3a040c4560 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c050e90 tx=0x7f3a0c0985b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcbdc1c32903dbce server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.869+0000 7fdf61a15640 1 --2- 172.21.15.5:0/4163366872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf5c14f8b0 0x7fdf5c145580 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c009a20 tx=0x7fdf4c01c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=870883d163eca51d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.869+0000 7f3a0b7fe640 1 -- 172.21.15.5:0/698750453 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c1234b0 con 0x7f3a0403a5b0 2024-03-18T15:33:48.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.869+0000 7fdf61214640 1 -- 172.21.15.5:0/4163366872 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c0a9620 con 0x7fdf5c14f8b0 2024-03-18T15:33:48.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7fdf61214640 1 -- 172.21.15.5:0/4163366872 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf4c0a97c0 con 0x7fdf5c14f8b0 2024-03-18T15:33:48.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7f3a0b7fe640 1 -- 172.21.15.5:0/698750453 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a0c123650 con 0x7f3a0403a5b0 2024-03-18T15:33:48.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7f3a0b7fe640 1 -- 172.21.15.5:0/698750453 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c123ac0 con 0x7f3a0403a5b0 2024-03-18T15:33:48.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7f5118467640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f51100b2240 0x7f5110148870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7f5118467640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5110149210 con 0x7f511013f5b0 2024-03-18T15:33:48.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7f5118467640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51100b2620 con 0x7f5110148db0 2024-03-18T15:33:48.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7f5118467640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f51100705a0 con 0x7f51100b2240 2024-03-18T15:33:48.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7f5116a04640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f511013f5b0 0x7f511014f9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:57288/0 (socket says 172.21.15.5:57288) 2024-03-18T15:33:48.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7f3a118b0640 1 -- 172.21.15.5:0/698750453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3a0403a5b0 msgr2=0x7f3a040c4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7f3a118b0640 1 --2- 172.21.15.5:0/698750453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3a0403a5b0 0x7f3a040c4560 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c050e90 tx=0x7f3a0c0985b0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7f5116a04640 1 -- 172.21.15.5:0/4016265839 learned_addr learned my addr 172.21.15.5:0/4016265839 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:33:48.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7fdf6447a640 1 -- 172.21.15.5:0/4163366872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf5c14f8b0 msgr2=0x7fdf5c145580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7f3a118b0640 1 -- 172.21.15.5:0/698750453 shutdown_connections 2024-03-18T15:33:48.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7f3a118b0640 1 --2- 172.21.15.5:0/698750453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3a0403a5b0 0x7f3a040c4560 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7f3a118b0640 1 --2- 172.21.15.5:0/698750453 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a04039080 0x7f3a040c4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7f3a118b0640 1 -- 172.21.15.5:0/698750453 >> 172.21.15.5:0/698750453 conn(0x7f3a0403cab0 msgr2=0x7f3a0403cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:48.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7f3a118b0640 1 -- 172.21.15.5:0/698750453 shutdown_connections 2024-03-18T15:33:48.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7f3a118b0640 1 -- 172.21.15.5:0/698750453 wait complete. 2024-03-18T15:33:48.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7fdf61214640 1 -- 172.21.15.5:0/4163366872 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c0a9620 con 0x7fdf5c14f8b0 2024-03-18T15:33:48.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7fdf6447a640 1 --2- 172.21.15.5:0/4163366872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf5c14f8b0 0x7fdf5c145580 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c009a20 tx=0x7fdf4c01c670 comp rx=0 tx=0).stop 2024-03-18T15:33:48.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7fdf6447a640 1 -- 172.21.15.5:0/4163366872 shutdown_connections 2024-03-18T15:33:48.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7fdf6447a640 1 --2- 172.21.15.5:0/4163366872 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdf5c0b1ce0 0x7fdf5c145ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7fdf6447a640 1 --2- 172.21.15.5:0/4163366872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf5c14f8b0 0x7fdf5c145580 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7fdf6447a640 1 -- 172.21.15.5:0/4163366872 >> 172.21.15.5:0/4163366872 conn(0x7fdf5c0673c0 msgr2=0x7fdf5c0677d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:48.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7f5116203640 1 --2- 172.21.15.5:0/4016265839 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f51100b2240 0x7f5110148870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7f5115201640 1 -- 172.21.15.5:0/4016265839 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3753608155 0 0) 0x7f5110149210 con 0x7f511013f5b0 2024-03-18T15:33:48.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7f5115201640 1 -- 172.21.15.5:0/4016265839 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f50fc003530 con 0x7f511013f5b0 2024-03-18T15:33:48.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7f5116203640 1 -- 172.21.15.5:0/4016265839 >> v1:172.21.15.80:6789/0 conn(0x7f5110148db0 legacy=0x7f511013eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7f5116203640 1 -- 172.21.15.5:0/4016265839 >> v1:172.21.15.5:6789/0 conn(0x7f511013f5b0 legacy=0x7f511014f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7f5116203640 1 -- 172.21.15.5:0/4016265839 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51100b3b70 con 0x7f51100b2240 2024-03-18T15:33:48.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7fdf6447a640 1 -- 172.21.15.5:0/4163366872 shutdown_connections 2024-03-18T15:33:48.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7f3a118b0640 1 Processor -- start 2024-03-18T15:33:48.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7f3a118b0640 1 -- start start 2024-03-18T15:33:48.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7f3a118b0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3a04038270 0x7f3a04006640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7f3a118b0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a04039080 0x7f3a04163e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7f3a118b0640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a0403a5b0 0x7f3a040b6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7f3a118b0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3a04039a10 con 0x7f3a04039080 2024-03-18T15:33:48.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7f3a118b0640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3a04039890 con 0x7f3a04038270 2024-03-18T15:33:48.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7f3a118b0640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f3a040d55c0 con 0x7f3a0403a5b0 2024-03-18T15:33:48.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7f3a110af640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a0403a5b0 0x7f3a040b6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7f3a110af640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a0403a5b0 0x7f3a040b6eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:53018/0 (socket says 172.21.15.5:53018) 2024-03-18T15:33:48.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7f3a110af640 1 -- 172.21.15.5:0/800028980 learned_addr learned my addr 172.21.15.5:0/800028980 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:48.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7fdf6447a640 1 -- 172.21.15.5:0/4163366872 wait complete. 2024-03-18T15:33:48.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7fdf6447a640 1 Processor -- start 2024-03-18T15:33:48.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7f3a110af640 1 -- 172.21.15.5:0/800028980 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3a04038270 msgr2=0x7f3a04006640 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T15:33:48.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7f3a110af640 1 --2- 172.21.15.5:0/800028980 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3a04038270 0x7f3a04006640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7f3a110af640 1 -- 172.21.15.5:0/800028980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a04039080 msgr2=0x7f3a04163e80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T15:33:48.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7f3a110af640 1 --2- 172.21.15.5:0/800028980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a04039080 0x7f3a04163e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7f3a110af640 1 -- 172.21.15.5:0/800028980 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a040c7410 con 0x7f3a0403a5b0 2024-03-18T15:33:48.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.877+0000 7f3a110af640 1 --2- 172.21.15.5:0/800028980 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a0403a5b0 0x7f3a040b6eb0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c02d750 tx=0x7f3a0c094990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.873+0000 7fdf6447a640 1 -- start start 2024-03-18T15:33:48.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.877+0000 7fdf6447a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf5c0b1ce0 0x7fdf5c1e05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.877+0000 7fdf6447a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdf5c0b9f40 0x7fdf5c1df860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.877+0000 7fdf6447a640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdf5c0bd6a0 0x7fdf5c1ec540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.877+0000 7fdf6447a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5c158af0 con 0x7fdf5c0b1ce0 2024-03-18T15:33:48.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.877+0000 7fdf6447a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fdf5c158970 con 0x7fdf5c0b9f40 2024-03-18T15:33:48.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.877+0000 7fdf6447a640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fdf5c158c70 con 0x7fdf5c0bd6a0 2024-03-18T15:33:48.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.877+0000 7fdf62216640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf5c0b1ce0 0x7fdf5c1e05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.877+0000 7fdf62216640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf5c0b1ce0 0x7fdf5c1e05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38560/0 (socket says 172.21.15.5:38560) 2024-03-18T15:33:48.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.877+0000 7fdf62216640 1 -- 172.21.15.5:0/3626767203 learned_addr learned my addr 172.21.15.5:0/3626767203 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:48.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.877+0000 7f5116203640 1 --2- 172.21.15.5:0/4016265839 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f51100b2240 0x7f5110148870 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f510c0097f0 tx=0x7f510c01c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34ec3307aa645040 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.877+0000 7fdf62a17640 1 --2- 172.21.15.5:0/3626767203 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdf5c0bd6a0 0x7fdf5c1ec540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.877+0000 7fdf62216640 1 -- 172.21.15.5:0/3626767203 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdf5c0bd6a0 msgr2=0x7fdf5c1ec540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.877+0000 7fdf62216640 1 --2- 172.21.15.5:0/3626767203 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdf5c0bd6a0 0x7fdf5c1ec540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.877+0000 7fdf62216640 1 -- 172.21.15.5:0/3626767203 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdf5c0b9f40 msgr2=0x7fdf5c1df860 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T15:33:48.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.877+0000 7fdf62216640 1 --2- 172.21.15.5:0/3626767203 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdf5c0b9f40 0x7fdf5c1df860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.877+0000 7fdf62216640 1 -- 172.21.15.5:0/3626767203 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf5c149ae0 con 0x7fdf5c0b1ce0 2024-03-18T15:33:48.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.877+0000 7fdf62a17640 1 --2- 172.21.15.5:0/3626767203 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdf5c0bd6a0 0x7fdf5c1ec540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:33:48.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.877+0000 7fdf62216640 1 --2- 172.21.15.5:0/3626767203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf5c0b1ce0 0x7fdf5c1e05c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdf5400cbc0 tx=0x7fdf5400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.877+0000 7fdf52ffd640 1 -- 172.21.15.5:0/3626767203 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf54017070 con 0x7fdf5c0b1ce0 2024-03-18T15:33:48.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.877+0000 7fdf52ffd640 1 -- 172.21.15.5:0/3626767203 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf5400aca0 con 0x7fdf5c0b1ce0 2024-03-18T15:33:48.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.877+0000 7fdf52ffd640 1 -- 172.21.15.5:0/3626767203 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf54012600 con 0x7fdf5c0b1ce0 2024-03-18T15:33:48.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.877+0000 7fdf6447a640 1 -- 172.21.15.5:0/3626767203 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf5c0bc010 con 0x7fdf5c0b1ce0 2024-03-18T15:33:48.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.877+0000 7fdf6447a640 1 -- 172.21.15.5:0/3626767203 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf5c149760 con 0x7fdf5c0b1ce0 2024-03-18T15:33:48.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.877+0000 7fdf6447a640 1 -- 172.21.15.5:0/3626767203 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf5c072930 con 0x7fdf5c0b1ce0 2024-03-18T15:33:48.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.881+0000 7fdf52ffd640 1 -- 172.21.15.5:0/3626767203 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fdf5401a070 con 0x7fdf5c0b1ce0 2024-03-18T15:33:48.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.881+0000 7fdf52ffd640 1 --2- 172.21.15.5:0/3626767203 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fdf400676b0 0x7fdf40069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.881+0000 7fdf52ffd640 1 -- 172.21.15.5:0/3626767203 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fdf54085990 con 0x7fdf5c0b1ce0 2024-03-18T15:33:48.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.881+0000 7fdf61a15640 1 --2- 172.21.15.5:0/3626767203 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fdf400676b0 0x7fdf40069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.881+0000 7fbd7effd640 1 -- 172.21.15.5:0/4170349154 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd780b2070 con 0x7fbd800b9230 2024-03-18T15:33:48.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.881+0000 7fbd7effd640 1 -- 172.21.15.5:0/4170349154 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd780a5800 con 0x7fbd800b9230 2024-03-18T15:33:48.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.881+0000 7fbd7effd640 1 -- 172.21.15.5:0/4170349154 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd780a5c10 con 0x7fbd800b9230 2024-03-18T15:33:48.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.881+0000 7f5115201640 1 -- 172.21.15.5:0/4016265839 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510c0ab620 con 0x7f51100b2240 2024-03-18T15:33:48.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.881+0000 7f5115201640 1 -- 172.21.15.5:0/4016265839 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f510c0b0070 con 0x7f51100b2240 2024-03-18T15:33:48.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.881+0000 7f5115201640 1 -- 172.21.15.5:0/4016265839 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510c0ab9c0 con 0x7f51100b2240 2024-03-18T15:33:48.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.881+0000 7f5118467640 1 -- 172.21.15.5:0/4016265839 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f51100b2240 msgr2=0x7f5110148870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.881+0000 7f5118467640 1 --2- 172.21.15.5:0/4016265839 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f51100b2240 0x7f5110148870 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f510c0097f0 tx=0x7f510c01c440 comp rx=0 tx=0).stop 2024-03-18T15:33:48.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.881+0000 7fdf61a15640 1 --2- 172.21.15.5:0/3626767203 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fdf400676b0 0x7fdf40069b70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c0ae000 tx=0x7fdf4c0b3040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.881+0000 7f0a957fa640 1 -- 172.21.15.5:0/3766413923 <== mon.2 v2:172.21.15.80:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f0a900c6030 con 0x7f0a98145250 2024-03-18T15:33:48.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.881+0000 7f3a097fa640 1 -- 172.21.15.5:0/800028980 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c1234b0 con 0x7f3a0403a5b0 2024-03-18T15:33:48.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.881+0000 7f3a097fa640 1 -- 172.21.15.5:0/800028980 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a0c12b810 con 0x7f3a0403a5b0 2024-03-18T15:33:48.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.881+0000 7f3a097fa640 1 -- 172.21.15.5:0/800028980 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0c12bbc0 con 0x7f3a0403a5b0 2024-03-18T15:33:48.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.881+0000 7f3a118b0640 1 -- 172.21.15.5:0/800028980 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a040c66a0 con 0x7f3a0403a5b0 2024-03-18T15:33:48.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.881+0000 7f3a118b0640 1 -- 172.21.15.5:0/800028980 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a040c63c0 con 0x7f3a0403a5b0 2024-03-18T15:33:48.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.885+0000 7f5118467640 1 -- 172.21.15.5:0/4016265839 shutdown_connections 2024-03-18T15:33:48.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.885+0000 7f5118467640 1 --2- 172.21.15.5:0/4016265839 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f51100b2240 0x7f5110148870 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.885+0000 7f5118467640 1 -- 172.21.15.5:0/4016265839 >> 172.21.15.5:0/4016265839 conn(0x7f51100b8aa0 msgr2=0x7f51100b8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:48.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.885+0000 7f3a118b0640 1 -- 172.21.15.5:0/800028980 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a04005720 con 0x7f3a0403a5b0 2024-03-18T15:33:48.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.885+0000 7fbd8696c640 1 -- 172.21.15.5:0/4170349154 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbd800b9230 msgr2=0x7fbd800b31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.885+0000 7fbd8696c640 1 --2- 172.21.15.5:0/4170349154 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbd800b9230 0x7fbd800b31c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbd78009d90 tx=0x7fbd7801c870 comp rx=0 tx=0).stop 2024-03-18T15:33:48.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.885+0000 7fbd8696c640 1 -- 172.21.15.5:0/4170349154 shutdown_connections 2024-03-18T15:33:48.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.885+0000 7fbd8696c640 1 --2- 172.21.15.5:0/4170349154 >> v2:172.21.15.80:3300/0 conn(0x7fbd800b43d0 0x7fbd800b4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.885+0000 7fbd8696c640 1 --2- 172.21.15.5:0/4170349154 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbd800b9230 0x7fbd800b31c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.885+0000 7fbd8696c640 1 -- 172.21.15.5:0/4170349154 >> 172.21.15.5:0/4170349154 conn(0x7fbd800675c0 msgr2=0x7fbd800bcc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:48.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.885+0000 7f5118467640 1 -- 172.21.15.5:0/4016265839 shutdown_connections 2024-03-18T15:33:48.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.885+0000 7f3a097fa640 1 -- 172.21.15.5:0/800028980 <== mon.2 v2:172.21.15.80:3301/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f3a0c13c060 con 0x7f3a0403a5b0 2024-03-18T15:33:48.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.885+0000 7fbd8696c640 1 -- 172.21.15.5:0/4170349154 shutdown_connections 2024-03-18T15:33:48.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.885+0000 7f3a097fa640 1 --2- 172.21.15.5:0/800028980 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f39f0067660 0x7f39f0069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.885+0000 7f3a097fa640 1 -- 172.21.15.5:0/800028980 <== mon.2 v2:172.21.15.80:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3a0c19a780 con 0x7f3a0403a5b0 2024-03-18T15:33:48.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.885+0000 7f5118467640 1 -- 172.21.15.5:0/4016265839 wait complete. 2024-03-18T15:33:48.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.885+0000 7f5118467640 1 Processor -- start 2024-03-18T15:33:48.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.885+0000 7fbd8696c640 1 -- 172.21.15.5:0/4170349154 wait complete. 2024-03-18T15:33:48.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.885+0000 7fbd8696c640 1 Processor -- start 2024-03-18T15:33:48.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.885+0000 7f5118467640 1 -- start start 2024-03-18T15:33:48.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.885+0000 7f5118467640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f51100b2240 0x7f51101ec010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.885+0000 7f5118467640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f511013f5b0 0x7f511013fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.885+0000 7f5118467640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5110148db0 0x7f5110140590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.885+0000 7f5118467640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f51100b45f0 con 0x7f511013f5b0 2024-03-18T15:33:48.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.885+0000 7f5118467640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f51100b4470 con 0x7f5110148db0 2024-03-18T15:33:48.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.885+0000 7f5118467640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f51100b4f40 con 0x7f51100b2240 2024-03-18T15:33:48.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.885+0000 7f5115a02640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f511013f5b0 0x7f511013fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.885+0000 7f5116203640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f51100b2240 0x7f51101ec010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.885+0000 7f5115a02640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f511013f5b0 0x7f511013fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38566/0 (socket says 172.21.15.5:38566) 2024-03-18T15:33:48.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.885+0000 7f5115a02640 1 -- 172.21.15.5:0/2167376199 learned_addr learned my addr 172.21.15.5:0/2167376199 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:48.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.885+0000 7f5116a04640 1 --2- 172.21.15.5:0/2167376199 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5110148db0 0x7f5110140590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.885+0000 7f3a108ae640 1 --2- 172.21.15.5:0/800028980 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f39f0067660 0x7f39f0069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.889+0000 7f5115a02640 1 -- 172.21.15.5:0/2167376199 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f51100b2240 msgr2=0x7f51101ec010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.889+0000 7f5115a02640 1 --2- 172.21.15.5:0/2167376199 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f51100b2240 0x7f51101ec010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.889+0000 7f5115a02640 1 -- 172.21.15.5:0/2167376199 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5110148db0 msgr2=0x7f5110140590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.889+0000 7f5115a02640 1 --2- 172.21.15.5:0/2167376199 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5110148db0 0x7f5110140590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.889+0000 7f5115a02640 1 -- 172.21.15.5:0/2167376199 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51101423b0 con 0x7f511013f5b0 2024-03-18T15:33:48.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.889+0000 7f3a108ae640 1 --2- 172.21.15.5:0/800028980 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f39f0067660 0x7f39f0069b20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f39f8007fd0 tx=0x7f39f8007900 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.889+0000 7f5115a02640 1 --2- 172.21.15.5:0/2167376199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f511013f5b0 0x7f511013fa70 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f510000c530 tx=0x7f510000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.889+0000 7fbd8696c640 1 -- start start 2024-03-18T15:33:48.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.889+0000 7fbd8696c640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbd800b43d0 0x7fbd801f0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.889+0000 7fbd8696c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd800bd5c0 0x7fbd801e6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.889+0000 7fbd8696c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbd80143d00 0x7fbd801e6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.889+0000 7fbd8696c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbd80154630 con 0x7fbd800bd5c0 2024-03-18T15:33:48.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.889+0000 7fbd8696c640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fbd80141d10 con 0x7fbd80143d00 2024-03-18T15:33:48.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.889+0000 7fbd8696c640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fbd801547b0 con 0x7fbd800b43d0 2024-03-18T15:33:48.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.889+0000 7fbd7f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd800bd5c0 0x7fbd801e6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.889+0000 7fbd84f09640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbd80143d00 0x7fbd801e6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.889+0000 7fbd7f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd800bd5c0 0x7fbd801e6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38574/0 (socket says 172.21.15.5:38574) 2024-03-18T15:33:48.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.889+0000 7fbd7f7fe640 1 -- 172.21.15.5:0/2328450136 learned_addr learned my addr 172.21.15.5:0/2328450136 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:48.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.889+0000 7fbd7ffff640 1 --2- 172.21.15.5:0/2328450136 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbd800b43d0 0x7fbd801f0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.889+0000 7fbd7f7fe640 1 -- 172.21.15.5:0/2328450136 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbd800b43d0 msgr2=0x7fbd801f0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.889+0000 7fbd7f7fe640 1 --2- 172.21.15.5:0/2328450136 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbd800b43d0 0x7fbd801f0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.889+0000 7fbd7f7fe640 1 -- 172.21.15.5:0/2328450136 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbd80143d00 msgr2=0x7fbd801e6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.889+0000 7fbd7f7fe640 1 --2- 172.21.15.5:0/2328450136 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbd80143d00 0x7fbd801e6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.889+0000 7fbd7f7fe640 1 -- 172.21.15.5:0/2328450136 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd80137460 con 0x7fbd800bd5c0 2024-03-18T15:33:48.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.889+0000 7fbd7f7fe640 1 --2- 172.21.15.5:0/2328450136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd800bd5c0 0x7fbd801e6830 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbd7801cd50 tx=0x7fbd78019610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.889+0000 7fdf52ffd640 1 -- 172.21.15.5:0/3626767203 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fdf540560a0 con 0x7fdf5c0b1ce0 2024-03-18T15:33:48.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.893+0000 7f5106ffd640 1 -- 172.21.15.5:0/2167376199 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5100016020 con 0x7f511013f5b0 2024-03-18T15:33:48.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.893+0000 7f5118467640 1 -- 172.21.15.5:0/2167376199 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5110142690 con 0x7f511013f5b0 2024-03-18T15:33:48.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.893+0000 7f5118467640 1 -- 172.21.15.5:0/2167376199 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51100b1d20 con 0x7f511013f5b0 2024-03-18T15:33:48.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.893+0000 7fbd7cff9640 1 -- 172.21.15.5:0/2328450136 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd780bb070 con 0x7fbd800bd5c0 2024-03-18T15:33:48.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.893+0000 7f5106ffd640 1 -- 172.21.15.5:0/2167376199 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5100005220 con 0x7f511013f5b0 2024-03-18T15:33:48.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.893+0000 7fbd8696c640 1 -- 172.21.15.5:0/2328450136 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd801f0a10 con 0x7fbd800bd5c0 2024-03-18T15:33:48.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.893+0000 7fbd8696c640 1 -- 172.21.15.5:0/2328450136 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd80137180 con 0x7fbd800bd5c0 2024-03-18T15:33:48.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.893+0000 7f5106ffd640 1 -- 172.21.15.5:0/2167376199 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5100007030 con 0x7f511013f5b0 2024-03-18T15:33:48.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.893+0000 7fbd7cff9640 1 -- 172.21.15.5:0/2328450136 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd780a5800 con 0x7fbd800bd5c0 2024-03-18T15:33:48.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.893+0000 7fbd7cff9640 1 -- 172.21.15.5:0/2328450136 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd780b2040 con 0x7fbd800bd5c0 2024-03-18T15:33:48.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.893+0000 7fbd8696c640 1 -- 172.21.15.5:0/2328450136 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd600052a0 con 0x7fbd800bd5c0 2024-03-18T15:33:48.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.893+0000 7f5106ffd640 1 -- 172.21.15.5:0/2167376199 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f51000071d0 con 0x7f511013f5b0 2024-03-18T15:33:48.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.897+0000 7f5106ffd640 1 --2- 172.21.15.5:0/2167376199 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f50f4067660 0x7f50f4069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.897+0000 7f5116203640 1 --2- 172.21.15.5:0/2167376199 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f50f4067660 0x7f50f4069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.897+0000 7fbd7cff9640 1 -- 172.21.15.5:0/2328450136 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fbd780ad440 con 0x7fbd800bd5c0 2024-03-18T15:33:48.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.897+0000 7f5116203640 1 --2- 172.21.15.5:0/2167376199 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f50f4067660 0x7f50f4069b20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f510c0af000 tx=0x7f510c0b6040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.897+0000 7fbd7cff9640 1 --2- 172.21.15.5:0/2328450136 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fbd6c067660 0x7fbd6c069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.897+0000 7f5106ffd640 1 -- 172.21.15.5:0/2167376199 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f51000537a0 con 0x7f511013f5b0 2024-03-18T15:33:48.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.897+0000 7fbd7ffff640 1 --2- 172.21.15.5:0/2328450136 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fbd6c067660 0x7fbd6c069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.897+0000 7fbd7cff9640 1 -- 172.21.15.5:0/2328450136 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fbd780eee50 con 0x7fbd800bd5c0 2024-03-18T15:33:48.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.897+0000 7fbd7ffff640 1 --2- 172.21.15.5:0/2328450136 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fbd6c067660 0x7fbd6c069b20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbd740011f0 tx=0x7fbd74009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.901+0000 7f3a097fa640 1 -- 172.21.15.5:0/800028980 <== mon.2 v2:172.21.15.80:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f3a0c127030 con 0x7f3a0403a5b0 2024-03-18T15:33:48.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.901+0000 7f5118467640 1 -- 172.21.15.5:0/2167376199 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50f80052a0 con 0x7f511013f5b0 2024-03-18T15:33:48.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.909+0000 7fbd7cff9640 1 -- 172.21.15.5:0/2328450136 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fbd78016580 con 0x7fbd800bd5c0 2024-03-18T15:33:48.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.921+0000 7f5106ffd640 1 -- 172.21.15.5:0/2167376199 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f5100020020 con 0x7f511013f5b0 2024-03-18T15:33:48.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.921+0000 7fe8757d3640 1 Processor -- start 2024-03-18T15:33:48.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.921+0000 7fe8757d3640 1 -- start start 2024-03-18T15:33:48.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.921+0000 7fe8757d3640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe8700b1c90 0x7fe8700ae3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.921+0000 7fe8757d3640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe870157f50 0x7fe87015eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.921+0000 7fe8757d3640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8700b2ac0 con 0x7fe8700b26e0 2024-03-18T15:33:48.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.921+0000 7fe8757d3640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fe8700b1800 con 0x7fe8700b1c90 2024-03-18T15:33:48.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.921+0000 7fe8757d3640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fe870072bd0 con 0x7fe870157f50 2024-03-18T15:33:48.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.921+0000 7fe86f7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe8700b1c90 0x7fe8700ae3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.921+0000 7fe86ffff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe8700b26e0 0x7fe870157840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:57298/0 (socket says 172.21.15.5:57298) 2024-03-18T15:33:48.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.921+0000 7fe86ffff640 1 -- 172.21.15.5:0/1261937304 learned_addr learned my addr 172.21.15.5:0/1261937304 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:33:48.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.925+0000 7fe86f7fe640 1 -- 172.21.15.5:0/1261937304 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe870157f50 msgr2=0x7fe87015eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:48.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.921+0000 7fe874fd2640 1 --2- 172.21.15.5:0/1261937304 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe870157f50 0x7fe87015eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.925+0000 7fe86f7fe640 1 --2- 172.21.15.5:0/1261937304 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe870157f50 0x7fe87015eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.925+0000 7fe86f7fe640 1 -- 172.21.15.5:0/1261937304 >> v1:172.21.15.5:6789/0 conn(0x7fe8700b26e0 legacy=0x7fe870157840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.925+0000 7fe86f7fe640 1 -- 172.21.15.5:0/1261937304 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8700b01a0 con 0x7fe8700b1c90 2024-03-18T15:33:48.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.925+0000 7fe874fd2640 1 --2- 172.21.15.5:0/1261937304 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe870157f50 0x7fe87015eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:33:48.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.921+0000 7fe86effd640 1 -- 172.21.15.5:0/1261937304 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3715128325 0 0) 0x7fe8700b2ac0 con 0x7fe8700b26e0 2024-03-18T15:33:48.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.929+0000 7fe86f7fe640 1 --2- 172.21.15.5:0/1261937304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8700b1c90 0x7fe8700ae3d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe868008d70 tx=0x7fe86801cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70fc1b3faad31afe server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.929+0000 7fe86effd640 1 -- 172.21.15.5:0/1261937304 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8680b2070 con 0x7fe8700b1c90 2024-03-18T15:33:48.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.937+0000 7fe86effd640 1 -- 172.21.15.5:0/1261937304 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8680a5b90 con 0x7fe8700b1c90 2024-03-18T15:33:48.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.937+0000 7fe8757d3640 1 -- 172.21.15.5:0/1261937304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8700b1c90 msgr2=0x7fe8700ae3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.937+0000 7fe8757d3640 1 --2- 172.21.15.5:0/1261937304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8700b1c90 0x7fe8700ae3d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe868008d70 tx=0x7fe86801cc30 comp rx=0 tx=0).stop 2024-03-18T15:33:48.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.937+0000 7fe8757d3640 1 -- 172.21.15.5:0/1261937304 shutdown_connections 2024-03-18T15:33:48.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.937+0000 7fe8757d3640 1 --2- 172.21.15.5:0/1261937304 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe870157f50 0x7fe87015eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.937+0000 7fe8757d3640 1 --2- 172.21.15.5:0/1261937304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8700b1c90 0x7fe8700ae3d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.937+0000 7fe8757d3640 1 -- 172.21.15.5:0/1261937304 >> 172.21.15.5:0/1261937304 conn(0x7fe8700b6a00 msgr2=0x7fe8700b6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:48.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.937+0000 7fe8757d3640 1 -- 172.21.15.5:0/1261937304 shutdown_connections 2024-03-18T15:33:48.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.937+0000 7fe8757d3640 1 -- 172.21.15.5:0/1261937304 wait complete. 2024-03-18T15:33:48.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.937+0000 7fe8757d3640 1 Processor -- start 2024-03-18T15:33:48.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.937+0000 7fe8757d3640 1 -- start start 2024-03-18T15:33:48.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.941+0000 7fe8757d3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8700b26e0 0x7fe870157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.941+0000 7fe8757d3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe870157f50 0x7fe87014d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.941+0000 7fe8757d3640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8701efbc0 0x7fe8701f0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.941+0000 7fe8757d3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe8700af690 con 0x7fe8700b26e0 2024-03-18T15:33:48.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.941+0000 7fe8757d3640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe8700afa40 con 0x7fe870157f50 2024-03-18T15:33:48.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.941+0000 7fe8757d3640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fe8700af810 con 0x7fe8701efbc0 2024-03-18T15:33:48.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.941+0000 7fe86ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8700b26e0 0x7fe870157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.941+0000 7fe86ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8700b26e0 0x7fe870157910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38590/0 (socket says 172.21.15.5:38590) 2024-03-18T15:33:48.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.941+0000 7fe86ffff640 1 -- 172.21.15.5:0/1289299972 learned_addr learned my addr 172.21.15.5:0/1289299972 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:48.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.941+0000 7fe874fd2640 1 --2- 172.21.15.5:0/1289299972 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8701efbc0 0x7fe8701f0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.941+0000 7fe86f7fe640 1 --2- 172.21.15.5:0/1289299972 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe870157f50 0x7fe87014d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.941+0000 7fe86ffff640 1 -- 172.21.15.5:0/1289299972 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8701efbc0 msgr2=0x7fe8701f0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.941+0000 7fe86ffff640 1 --2- 172.21.15.5:0/1289299972 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8701efbc0 0x7fe8701f0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.941+0000 7fe86ffff640 1 -- 172.21.15.5:0/1289299972 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe870157f50 msgr2=0x7fe87014d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.941+0000 7fe86ffff640 1 --2- 172.21.15.5:0/1289299972 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe870157f50 0x7fe87014d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.941+0000 7fe86ffff640 1 -- 172.21.15.5:0/1289299972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8701f0590 con 0x7fe8700b26e0 2024-03-18T15:33:48.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.941+0000 7fe86ffff640 1 --2- 172.21.15.5:0/1289299972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8700b26e0 0x7fe870157910 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe86000c4d0 tx=0x7fe86000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.941+0000 7f6b91a18640 1 Processor -- start 2024-03-18T15:33:48.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.941+0000 7fe86cff9640 1 -- 172.21.15.5:0/1289299972 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe860018070 con 0x7fe8700b26e0 2024-03-18T15:33:48.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.941+0000 7fe86cff9640 1 -- 172.21.15.5:0/1289299972 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8600074e0 con 0x7fe8700b26e0 2024-03-18T15:33:48.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.941+0000 7fe8757d3640 1 -- 172.21.15.5:0/1289299972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe87014db90 con 0x7fe8700b26e0 2024-03-18T15:33:48.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.941+0000 7fe8757d3640 1 -- 172.21.15.5:0/1289299972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe870150540 con 0x7fe8700b26e0 2024-03-18T15:33:48.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.941+0000 7fe86cff9640 1 -- 172.21.15.5:0/1289299972 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe860004a30 con 0x7fe8700b26e0 2024-03-18T15:33:48.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.941+0000 7fe8757d3640 1 -- 172.21.15.5:0/1289299972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8500052a0 con 0x7fe8700b26e0 2024-03-18T15:33:48.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.953+0000 7fe86cff9640 1 -- 172.21.15.5:0/1289299972 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fe86001b070 con 0x7fe8700b26e0 2024-03-18T15:33:48.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.953+0000 7fe86cff9640 1 --2- 172.21.15.5:0/1289299972 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe85c067730 0x7fe85c069bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.953+0000 7fe86cff9640 1 -- 172.21.15.5:0/1289299972 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe860085080 con 0x7fe8700b26e0 2024-03-18T15:33:48.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.953+0000 7fe86f7fe640 1 --2- 172.21.15.5:0/1289299972 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe85c067730 0x7fe85c069bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.953+0000 7fe86f7fe640 1 --2- 172.21.15.5:0/1289299972 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe85c067730 0x7fe85c069bf0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe8680af000 tx=0x7fe8680a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.953+0000 7f6b91a18640 1 -- start start 2024-03-18T15:33:48.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.953+0000 7fe86cff9640 1 -- 172.21.15.5:0/1289299972 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fe860022020 con 0x7fe8700b26e0 2024-03-18T15:33:48.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.953+0000 7f6b91a18640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6b8c0b2bc0 0x7f6b8c0b5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.953+0000 7f6b91a18640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6b8c143900 0x7f6b8c143d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.953+0000 7f6b91a18640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b8c147720 con 0x7f6b8c06d990 2024-03-18T15:33:48.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.953+0000 7f6b91a18640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f6b8c0b4ca0 con 0x7f6b8c143900 2024-03-18T15:33:48.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.953+0000 7f6b91a18640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f6b8c0b0240 con 0x7f6b8c0b2bc0 2024-03-18T15:33:48.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.953+0000 7f6b90a16640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f6b8c06d990 0x7f6b8c147010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:57306/0 (socket says 172.21.15.5:57306) 2024-03-18T15:33:48.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.953+0000 7f6b90a16640 1 -- 172.21.15.5:0/3900546381 learned_addr learned my addr 172.21.15.5:0/3900546381 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:33:48.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.953+0000 7f6b8bfff640 1 --2- 172.21.15.5:0/3900546381 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6b8c0b2bc0 0x7f6b8c0b5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.957+0000 7f6b8bfff640 1 -- 172.21.15.5:0/3900546381 >> v2:172.21.15.5:3300/0 conn(0x7f6b8c143900 msgr2=0x7f6b8c143d90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-18T15:33:48.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.957+0000 7f6b8bfff640 1 --2- 172.21.15.5:0/3900546381 >> v2:172.21.15.5:3300/0 conn(0x7f6b8c143900 0x7f6b8c143d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.957+0000 7f6b8bfff640 1 -- 172.21.15.5:0/3900546381 >> v1:172.21.15.5:6789/0 conn(0x7f6b8c06d990 legacy=0x7f6b8c147010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.957+0000 7f6b8bfff640 1 -- 172.21.15.5:0/3900546381 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b8c146810 con 0x7f6b8c0b2bc0 2024-03-18T15:33:48.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.957+0000 7f6b8bfff640 1 --2- 172.21.15.5:0/3900546381 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6b8c0b2bc0 0x7f6b8c0b5aa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6b84009d90 tx=0x7f6b8401c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f48be88a4894b79 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.961+0000 7f6b8b7fe640 1 -- 172.21.15.5:0/3900546381 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b840b2070 con 0x7f6b8c0b2bc0 2024-03-18T15:33:48.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.961+0000 7f6b8b7fe640 1 -- 172.21.15.5:0/3900546381 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b840a57f0 con 0x7f6b8c0b2bc0 2024-03-18T15:33:48.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.961+0000 7f6b91a18640 1 -- 172.21.15.5:0/3900546381 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6b8c0b2bc0 msgr2=0x7f6b8c0b5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.961+0000 7f6b91a18640 1 --2- 172.21.15.5:0/3900546381 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6b8c0b2bc0 0x7f6b8c0b5aa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6b84009d90 tx=0x7f6b8401c920 comp rx=0 tx=0).stop 2024-03-18T15:33:48.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.961+0000 7f6b91a18640 1 -- 172.21.15.5:0/3900546381 shutdown_connections 2024-03-18T15:33:48.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.961+0000 7f6b91a18640 1 --2- 172.21.15.5:0/3900546381 >> v2:172.21.15.5:3300/0 conn(0x7f6b8c143900 0x7f6b8c143d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.961+0000 7f6b91a18640 1 --2- 172.21.15.5:0/3900546381 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6b8c0b2bc0 0x7f6b8c0b5aa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.961+0000 7f6b91a18640 1 -- 172.21.15.5:0/3900546381 >> 172.21.15.5:0/3900546381 conn(0x7f6b8c06ce70 msgr2=0x7f6b8c06d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:48.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.961+0000 7f6b91a18640 1 -- 172.21.15.5:0/3900546381 shutdown_connections 2024-03-18T15:33:48.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.961+0000 7f6b91a18640 1 -- 172.21.15.5:0/3900546381 wait complete. 2024-03-18T15:33:48.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.961+0000 7f6b91a18640 1 Processor -- start 2024-03-18T15:33:48.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.961+0000 7f6b91a18640 1 -- start start 2024-03-18T15:33:48.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.965+0000 7f6b91a18640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6b8c06d990 0x7f6b8c13aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.965+0000 7f6b91a18640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6b8c0b2bc0 0x7f6b8c136e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.965+0000 7f6b91a18640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b8c143900 0x7f6b8c145d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.965+0000 7f6b91a18640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c0b5090 con 0x7f6b8c143900 2024-03-18T15:33:48.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.965+0000 7f6b91a18640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c14ec50 con 0x7f6b8c06d990 2024-03-18T15:33:48.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.965+0000 7f6b91a18640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f6b8c0b5210 con 0x7f6b8c0b2bc0 2024-03-18T15:33:48.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.965+0000 7f6b90a16640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6b8c06d990 0x7f6b8c13aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.965+0000 7f6b90a16640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6b8c06d990 0x7f6b8c13aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:59094/0 (socket says 172.21.15.5:59094) 2024-03-18T15:33:48.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.965+0000 7f6b90a16640 1 -- 172.21.15.5:0/4047817629 learned_addr learned my addr 172.21.15.5:0/4047817629 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:48.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.965+0000 7f6b8bfff640 1 --2- 172.21.15.5:0/4047817629 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6b8c0b2bc0 0x7f6b8c136e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.965+0000 7f6b90a16640 1 -- 172.21.15.5:0/4047817629 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6b8c0b2bc0 msgr2=0x7f6b8c136e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:48.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.965+0000 7f6b90a16640 1 --2- 172.21.15.5:0/4047817629 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6b8c0b2bc0 0x7f6b8c136e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.965+0000 7f6b90a16640 1 -- 172.21.15.5:0/4047817629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b8c143900 msgr2=0x7f6b8c145d10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-18T15:33:48.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.965+0000 7f6b91217640 1 --2- 172.21.15.5:0/4047817629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b8c143900 0x7f6b8c145d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.965+0000 7f6b90a16640 1 --2- 172.21.15.5:0/4047817629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b8c143900 0x7f6b8c145d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:48.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.965+0000 7f6b90a16640 1 -- 172.21.15.5:0/4047817629 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b8c139650 con 0x7f6b8c06d990 2024-03-18T15:33:48.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.965+0000 7f6b91217640 1 --2- 172.21.15.5:0/4047817629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b8c143900 0x7f6b8c145d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:33:48.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.965+0000 7f6b90a16640 1 --2- 172.21.15.5:0/4047817629 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6b8c06d990 0x7f6b8c13aed0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c00c500 tx=0x7f6b7c00c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.965+0000 7f6b897fa640 1 -- 172.21.15.5:0/4047817629 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c017070 con 0x7f6b8c06d990 2024-03-18T15:33:48.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.965+0000 7f6b91a18640 1 -- 172.21.15.5:0/4047817629 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b8c13cd60 con 0x7f6b8c06d990 2024-03-18T15:33:48.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.965+0000 7f6b91a18640 1 -- 172.21.15.5:0/4047817629 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b8c13d320 con 0x7f6b8c06d990 2024-03-18T15:33:48.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.969+0000 7f6b897fa640 1 -- 172.21.15.5:0/4047817629 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b7c005cf0 con 0x7f6b8c06d990 2024-03-18T15:33:48.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.969+0000 7f6b897fa640 1 -- 172.21.15.5:0/4047817629 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c0072f0 con 0x7f6b8c06d990 2024-03-18T15:33:48.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.969+0000 7f6b897fa640 1 -- 172.21.15.5:0/4047817629 <== mon.1 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f6b7c01a070 con 0x7f6b8c06d990 2024-03-18T15:33:48.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.969+0000 7f6b897fa640 1 --2- 172.21.15.5:0/4047817629 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6b740678d0 0x7f6b74069d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:48.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.969+0000 7f6b8bfff640 1 --2- 172.21.15.5:0/4047817629 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6b740678d0 0x7f6b74069d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:48.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.969+0000 7f6b897fa640 1 -- 172.21.15.5:0/4047817629 <== mon.1 v2:172.21.15.80:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f6b7c084c60 con 0x7f6b8c06d990 2024-03-18T15:33:48.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.973+0000 7f6b8bfff640 1 --2- 172.21.15.5:0/4047817629 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6b740678d0 0x7f6b74069d90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6b840af000 tx=0x7f6b840a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:48.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.981+0000 7f6b91a18640 1 -- 172.21.15.5:0/4047817629 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b6c0052a0 con 0x7f6b8c06d990 2024-03-18T15:33:48.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:48.985+0000 7f6b897fa640 1 -- 172.21.15.5:0/4047817629 <== mon.1 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f6b8c147720 con 0x7f6b8c06d990 2024-03-18T15:33:49.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.181+0000 7f887910a640 1 -- 172.21.15.5:0/2278470046 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f8854005860 con 0x7f88740b1160 2024-03-18T15:33:49.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.181+0000 7f885b7fe640 1 -- 172.21.15.5:0/2278470046 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8864056990 con 0x7f88740b1160 2024-03-18T15:33:49.186 INFO:teuthology.orchestra.run.smithi005.stdout:60129542151 2024-03-18T15:33:49.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.185+0000 7f0a9e35e640 1 -- 172.21.15.5:0/3766413923 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f0a98143f90 con 0x7f0a98145250 2024-03-18T15:33:49.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.185+0000 7f887910a640 1 -- 172.21.15.5:0/2278470046 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f88600676b0 msgr2=0x7f8860069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:49.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.185+0000 7f887910a640 1 --2- 172.21.15.5:0/2278470046 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f88600676b0 0x7f8860069b70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f88740af910 tx=0x7f886c009040 comp rx=0 tx=0).stop 2024-03-18T15:33:49.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.185+0000 7f887910a640 1 -- 172.21.15.5:0/2278470046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88740b1160 msgr2=0x7f88741e7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:49.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.185+0000 7f887910a640 1 --2- 172.21.15.5:0/2278470046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88740b1160 0x7f88741e7c00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f886400d690 tx=0x7f886400db50 comp rx=0 tx=0).stop 2024-03-18T15:33:49.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.185+0000 7f0a957fa640 1 -- 172.21.15.5:0/3766413923 <== mon.2 v2:172.21.15.80:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0a90016620 con 0x7f0a98145250 2024-03-18T15:33:49.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.185+0000 7f887910a640 1 -- 172.21.15.5:0/2278470046 shutdown_connections 2024-03-18T15:33:49.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.185+0000 7f887910a640 1 --2- 172.21.15.5:0/2278470046 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f88600676b0 0x7f8860069b70 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.185+0000 7f887910a640 1 --2- 172.21.15.5:0/2278470046 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f88740b4ee0 0x7f88740718b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.185+0000 7f887910a640 1 --2- 172.21.15.5:0/2278470046 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f88740b1540 0x7f8874136db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.185+0000 7f887910a640 1 --2- 172.21.15.5:0/2278470046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f88740b1160 0x7f88741e7c00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.185+0000 7f887910a640 1 -- 172.21.15.5:0/2278470046 >> 172.21.15.5:0/2278470046 conn(0x7f8874066fc0 msgr2=0x7f88741433d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:49.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.185+0000 7f887910a640 1 -- 172.21.15.5:0/2278470046 shutdown_connections 2024-03-18T15:33:49.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.185+0000 7f887910a640 1 -- 172.21.15.5:0/2278470046 wait complete. 2024-03-18T15:33:49.192 INFO:teuthology.orchestra.run.smithi005.stdout:60129542150 2024-03-18T15:33:49.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.189+0000 7f0a9e35e640 1 -- 172.21.15.5:0/3766413923 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0a78067760 msgr2=0x7f0a78069c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:49.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.189+0000 7f0a9e35e640 1 --2- 172.21.15.5:0/3766413923 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0a78067760 0x7f0a78069c20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0a80006fd0 tx=0x7f0a80008040 comp rx=0 tx=0).stop 2024-03-18T15:33:49.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.189+0000 7f0a9e35e640 1 -- 172.21.15.5:0/3766413923 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a98145250 msgr2=0x7f0a981d7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:49.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.189+0000 7f0a9e35e640 1 --2- 172.21.15.5:0/3766413923 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a98145250 0x7f0a981d7e30 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0a9001a540 tx=0x7f0a900196b0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.189+0000 7f0a9e35e640 1 -- 172.21.15.5:0/3766413923 shutdown_connections 2024-03-18T15:33:49.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.189+0000 7f0a9e35e640 1 --2- 172.21.15.5:0/3766413923 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0a78067760 0x7f0a78069c20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.189+0000 7f0a9e35e640 1 --2- 172.21.15.5:0/3766413923 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a98145250 0x7f0a981d7e30 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.189+0000 7f0a9e35e640 1 --2- 172.21.15.5:0/3766413923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a980b7c30 0x7f0a981d78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.189+0000 7f0a9e35e640 1 --2- 172.21.15.5:0/3766413923 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a980b7850 0x7f0a98071430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.189+0000 7f0a9e35e640 1 -- 172.21.15.5:0/3766413923 >> 172.21.15.5:0/3766413923 conn(0x7f0a98067190 msgr2=0x7f0a980690d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:49.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.193+0000 7f0a9e35e640 1 -- 172.21.15.5:0/3766413923 shutdown_connections 2024-03-18T15:33:49.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.193+0000 7f0a9e35e640 1 -- 172.21.15.5:0/3766413923 wait complete. 2024-03-18T15:33:49.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.197+0000 7fdf6447a640 1 -- 172.21.15.5:0/3626767203 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fdf5c04f550 con 0x7fdf5c0b1ce0 2024-03-18T15:33:49.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.197+0000 7fdf52ffd640 1 -- 172.21.15.5:0/3626767203 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdf5405a140 con 0x7fdf5c0b1ce0 2024-03-18T15:33:49.203 INFO:teuthology.orchestra.run.smithi005.stdout:64424509447 2024-03-18T15:33:49.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.205+0000 7fdf50ff9640 1 -- 172.21.15.5:0/3626767203 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fdf400676b0 msgr2=0x7fdf40069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:49.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.205+0000 7fdf50ff9640 1 --2- 172.21.15.5:0/3626767203 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fdf400676b0 0x7fdf40069b70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c0ae000 tx=0x7fdf4c0b3040 comp rx=0 tx=0).stop 2024-03-18T15:33:49.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.205+0000 7fdf50ff9640 1 -- 172.21.15.5:0/3626767203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf5c0b1ce0 msgr2=0x7fdf5c1e05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:49.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.205+0000 7fdf50ff9640 1 --2- 172.21.15.5:0/3626767203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf5c0b1ce0 0x7fdf5c1e05c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdf5400cbc0 tx=0x7fdf5400a060 comp rx=0 tx=0).stop 2024-03-18T15:33:49.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.205+0000 7fdf50ff9640 1 -- 172.21.15.5:0/3626767203 shutdown_connections 2024-03-18T15:33:49.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.205+0000 7fdf50ff9640 1 --2- 172.21.15.5:0/3626767203 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fdf400676b0 0x7fdf40069b70 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.205+0000 7fdf50ff9640 1 --2- 172.21.15.5:0/3626767203 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdf5c0bd6a0 0x7fdf5c1ec540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.205+0000 7fdf50ff9640 1 --2- 172.21.15.5:0/3626767203 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdf5c0b9f40 0x7fdf5c1df860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.205+0000 7fdf50ff9640 1 --2- 172.21.15.5:0/3626767203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf5c0b1ce0 0x7fdf5c1e05c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.205+0000 7fdf50ff9640 1 -- 172.21.15.5:0/3626767203 >> 172.21.15.5:0/3626767203 conn(0x7fdf5c0673c0 msgr2=0x7fdf5c0740f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:49.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.205+0000 7fdf50ff9640 1 -- 172.21.15.5:0/3626767203 shutdown_connections 2024-03-18T15:33:49.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.205+0000 7fdf50ff9640 1 -- 172.21.15.5:0/3626767203 wait complete. 2024-03-18T15:33:49.214 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.7 2024-03-18T15:33:49.214 DEBUG:teuthology.parallel:result is None 2024-03-18T15:33:49.221 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.3 2024-03-18T15:33:49.221 DEBUG:teuthology.parallel:result is None 2024-03-18T15:33:49.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.229+0000 7fbd8696c640 1 -- 172.21.15.5:0/2328450136 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fbd60005860 con 0x7fbd800bd5c0 2024-03-18T15:33:49.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.229+0000 7fbd7cff9640 1 -- 172.21.15.5:0/2328450136 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbd780f5e20 con 0x7fbd800bd5c0 2024-03-18T15:33:49.231 INFO:teuthology.orchestra.run.smithi005.stdout:60129542150 2024-03-18T15:33:49.232 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.6 2024-03-18T15:33:49.232 DEBUG:teuthology.parallel:result is None 2024-03-18T15:33:49.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.233+0000 7fbd8696c640 1 -- 172.21.15.5:0/2328450136 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fbd6c067660 msgr2=0x7fbd6c069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:49.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.233+0000 7fbd8696c640 1 --2- 172.21.15.5:0/2328450136 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fbd6c067660 0x7fbd6c069b20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fbd740011f0 tx=0x7fbd74009040 comp rx=0 tx=0).stop 2024-03-18T15:33:49.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.233+0000 7fbd8696c640 1 -- 172.21.15.5:0/2328450136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd800bd5c0 msgr2=0x7fbd801e6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:49.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.233+0000 7fbd8696c640 1 --2- 172.21.15.5:0/2328450136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd800bd5c0 0x7fbd801e6830 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbd7801cd50 tx=0x7fbd78019610 comp rx=0 tx=0).stop 2024-03-18T15:33:49.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.233+0000 7fbd8696c640 1 -- 172.21.15.5:0/2328450136 shutdown_connections 2024-03-18T15:33:49.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.233+0000 7fbd8696c640 1 --2- 172.21.15.5:0/2328450136 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fbd6c067660 0x7fbd6c069b20 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.233+0000 7fbd8696c640 1 --2- 172.21.15.5:0/2328450136 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbd80143d00 0x7fbd801e6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.233+0000 7fbd8696c640 1 --2- 172.21.15.5:0/2328450136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd800bd5c0 0x7fbd801e6830 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.233+0000 7fbd8696c640 1 --2- 172.21.15.5:0/2328450136 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbd800b43d0 0x7fbd801f0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.233+0000 7fbd8696c640 1 -- 172.21.15.5:0/2328450136 >> 172.21.15.5:0/2328450136 conn(0x7fbd800675c0 msgr2=0x7fbd801411c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:49.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.233+0000 7fbd8696c640 1 -- 172.21.15.5:0/2328450136 shutdown_connections 2024-03-18T15:33:49.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.233+0000 7fbd8696c640 1 -- 172.21.15.5:0/2328450136 wait complete. 2024-03-18T15:33:49.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.237+0000 7f5118467640 1 -- 172.21.15.5:0/2167376199 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f50f8005860 con 0x7f511013f5b0 2024-03-18T15:33:49.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.241+0000 7f5106ffd640 1 -- 172.21.15.5:0/2167376199 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f51000074c0 con 0x7f511013f5b0 2024-03-18T15:33:49.241 INFO:teuthology.orchestra.run.smithi005.stdout:60129542150 2024-03-18T15:33:49.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.241+0000 7f5118467640 1 -- 172.21.15.5:0/2167376199 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f50f4067660 msgr2=0x7f50f4069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:49.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.241+0000 7f5118467640 1 --2- 172.21.15.5:0/2167376199 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f50f4067660 0x7f50f4069b20 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f510c0af000 tx=0x7f510c0b6040 comp rx=0 tx=0).stop 2024-03-18T15:33:49.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.241+0000 7f5118467640 1 -- 172.21.15.5:0/2167376199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f511013f5b0 msgr2=0x7f511013fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:49.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.241+0000 7f5118467640 1 --2- 172.21.15.5:0/2167376199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f511013f5b0 0x7f511013fa70 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f510000c530 tx=0x7f510000c9f0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.241+0000 7f5118467640 1 -- 172.21.15.5:0/2167376199 shutdown_connections 2024-03-18T15:33:49.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.241+0000 7f5118467640 1 --2- 172.21.15.5:0/2167376199 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f50f4067660 0x7f50f4069b20 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.241+0000 7f5118467640 1 --2- 172.21.15.5:0/2167376199 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5110148db0 0x7f5110140590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.241+0000 7f5118467640 1 --2- 172.21.15.5:0/2167376199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f511013f5b0 0x7f511013fa70 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.241+0000 7f5118467640 1 --2- 172.21.15.5:0/2167376199 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f51100b2240 0x7f51101ec010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.241+0000 7f5118467640 1 -- 172.21.15.5:0/2167376199 >> 172.21.15.5:0/2167376199 conn(0x7f51100b8aa0 msgr2=0x7f511013ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:49.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.241+0000 7f5118467640 1 -- 172.21.15.5:0/2167376199 shutdown_connections 2024-03-18T15:33:49.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.241+0000 7f5118467640 1 -- 172.21.15.5:0/2167376199 wait complete. 2024-03-18T15:33:49.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.241+0000 7f3a118b0640 1 -- 172.21.15.5:0/800028980 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f3a040c8a90 con 0x7f3a0403a5b0 2024-03-18T15:33:49.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.241+0000 7f3a097fa640 1 -- 172.21.15.5:0/800028980 <== mon.2 v2:172.21.15.80:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3a0c091cf0 con 0x7f3a0403a5b0 2024-03-18T15:33:49.245 INFO:teuthology.orchestra.run.smithi005.stdout:64424509447 2024-03-18T15:33:49.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.245+0000 7f3a118b0640 1 -- 172.21.15.5:0/800028980 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f39f0067660 msgr2=0x7f39f0069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:49.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.245+0000 7f3a118b0640 1 --2- 172.21.15.5:0/800028980 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f39f0067660 0x7f39f0069b20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f39f8007fd0 tx=0x7f39f8007900 comp rx=0 tx=0).stop 2024-03-18T15:33:49.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.245+0000 7f3a118b0640 1 -- 172.21.15.5:0/800028980 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a0403a5b0 msgr2=0x7f3a040b6eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:49.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.245+0000 7f3a118b0640 1 --2- 172.21.15.5:0/800028980 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a0403a5b0 0x7f3a040b6eb0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c02d750 tx=0x7f3a0c094990 comp rx=0 tx=0).stop 2024-03-18T15:33:49.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.245+0000 7f3a118b0640 1 -- 172.21.15.5:0/800028980 shutdown_connections 2024-03-18T15:33:49.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.245+0000 7f3a118b0640 1 --2- 172.21.15.5:0/800028980 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f39f0067660 0x7f39f0069b20 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.245+0000 7f3a118b0640 1 --2- 172.21.15.5:0/800028980 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3a0403a5b0 0x7f3a040b6eb0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.245+0000 7f3a118b0640 1 --2- 172.21.15.5:0/800028980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a04039080 0x7f3a04163e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.245+0000 7f3a118b0640 1 --2- 172.21.15.5:0/800028980 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3a04038270 0x7f3a04006640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.245+0000 7f3a118b0640 1 -- 172.21.15.5:0/800028980 >> 172.21.15.5:0/800028980 conn(0x7f3a0403cab0 msgr2=0x7f3a040bfb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:49.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.245+0000 7f3a118b0640 1 -- 172.21.15.5:0/800028980 shutdown_connections 2024-03-18T15:33:49.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.245+0000 7f3a118b0640 1 -- 172.21.15.5:0/800028980 wait complete. 2024-03-18T15:33:49.259 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.1 2024-03-18T15:33:49.259 DEBUG:teuthology.parallel:result is None 2024-03-18T15:33:49.264 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.2 2024-03-18T15:33:49.264 DEBUG:teuthology.parallel:result is None 2024-03-18T15:33:49.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.265+0000 7fe8757d3640 1 -- 172.21.15.5:0/1289299972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fe850005860 con 0x7fe8700b26e0 2024-03-18T15:33:49.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.265+0000 7fe86cff9640 1 -- 172.21.15.5:0/1289299972 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe87014db90 con 0x7fe8700b26e0 2024-03-18T15:33:49.266 INFO:teuthology.orchestra.run.smithi005.stdout:60129542149 2024-03-18T15:33:49.266 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.5 2024-03-18T15:33:49.266 DEBUG:teuthology.parallel:result is None 2024-03-18T15:33:49.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.265+0000 7fe8757d3640 1 -- 172.21.15.5:0/1289299972 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe85c067730 msgr2=0x7fe85c069bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:49.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.265+0000 7fe8757d3640 1 --2- 172.21.15.5:0/1289299972 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe85c067730 0x7fe85c069bf0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe8680af000 tx=0x7fe8680a7070 comp rx=0 tx=0).stop 2024-03-18T15:33:49.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.265+0000 7fe8757d3640 1 -- 172.21.15.5:0/1289299972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8700b26e0 msgr2=0x7fe870157910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:49.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.265+0000 7fe8757d3640 1 --2- 172.21.15.5:0/1289299972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8700b26e0 0x7fe870157910 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe86000c4d0 tx=0x7fe86000c990 comp rx=0 tx=0).stop 2024-03-18T15:33:49.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.265+0000 7fe8757d3640 1 -- 172.21.15.5:0/1289299972 shutdown_connections 2024-03-18T15:33:49.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.265+0000 7fe8757d3640 1 --2- 172.21.15.5:0/1289299972 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe85c067730 0x7fe85c069bf0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.265+0000 7fe8757d3640 1 --2- 172.21.15.5:0/1289299972 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe8701efbc0 0x7fe8701f0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.265+0000 7fe8757d3640 1 --2- 172.21.15.5:0/1289299972 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe870157f50 0x7fe87014d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.265+0000 7fe8757d3640 1 --2- 172.21.15.5:0/1289299972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8700b26e0 0x7fe870157910 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.265+0000 7fe8757d3640 1 -- 172.21.15.5:0/1289299972 >> 172.21.15.5:0/1289299972 conn(0x7fe8700b6a00 msgr2=0x7fe8700b4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:49.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.265+0000 7fe8757d3640 1 -- 172.21.15.5:0/1289299972 shutdown_connections 2024-03-18T15:33:49.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.265+0000 7fe8757d3640 1 -- 172.21.15.5:0/1289299972 wait complete. 2024-03-18T15:33:49.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.277+0000 7f6b91a18640 1 -- 172.21.15.5:0/4047817629 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f6b6c005590 con 0x7f6b8c06d990 2024-03-18T15:33:49.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.277+0000 7f6b897fa640 1 -- 172.21.15.5:0/4047817629 <== mon.1 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6b7c055300 con 0x7f6b8c06d990 2024-03-18T15:33:49.281 INFO:teuthology.orchestra.run.smithi005.stdout:60129542152 2024-03-18T15:33:49.282 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.0 2024-03-18T15:33:49.283 DEBUG:teuthology.parallel:result is None 2024-03-18T15:33:49.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.281+0000 7f6b91a18640 1 -- 172.21.15.5:0/4047817629 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6b740678d0 msgr2=0x7f6b74069d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:49.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.281+0000 7f6b91a18640 1 --2- 172.21.15.5:0/4047817629 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6b740678d0 0x7f6b74069d90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6b840af000 tx=0x7f6b840a7070 comp rx=0 tx=0).stop 2024-03-18T15:33:49.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.281+0000 7f6b91a18640 1 -- 172.21.15.5:0/4047817629 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6b8c06d990 msgr2=0x7f6b8c13aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:49.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.281+0000 7f6b91a18640 1 --2- 172.21.15.5:0/4047817629 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6b8c06d990 0x7f6b8c13aed0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c00c500 tx=0x7f6b7c00c9c0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.281+0000 7f6b91a18640 1 -- 172.21.15.5:0/4047817629 shutdown_connections 2024-03-18T15:33:49.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.281+0000 7f6b91a18640 1 --2- 172.21.15.5:0/4047817629 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6b740678d0 0x7f6b74069d90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.281+0000 7f6b91a18640 1 --2- 172.21.15.5:0/4047817629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b8c143900 0x7f6b8c145d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.281+0000 7f6b91a18640 1 --2- 172.21.15.5:0/4047817629 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6b8c0b2bc0 0x7f6b8c136e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.281+0000 7f6b91a18640 1 --2- 172.21.15.5:0/4047817629 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6b8c06d990 0x7f6b8c13aed0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.281+0000 7f6b91a18640 1 -- 172.21.15.5:0/4047817629 >> 172.21.15.5:0/4047817629 conn(0x7f6b8c06ce70 msgr2=0x7f6b8c0690d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:49.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.281+0000 7f6b91a18640 1 -- 172.21.15.5:0/4047817629 shutdown_connections 2024-03-18T15:33:49.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.281+0000 7f6b91a18640 1 -- 172.21.15.5:0/4047817629 wait complete. 2024-03-18T15:33:49.297 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542152 for osd.4 2024-03-18T15:33:49.297 DEBUG:teuthology.parallel:result is None 2024-03-18T15:33:49.297 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-18T15:33:49.298 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-18T15:33:49.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.445+0000 7f6cdc357640 1 Processor -- start 2024-03-18T15:33:49.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.445+0000 7f6cdc357640 1 -- start start 2024-03-18T15:33:49.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.445+0000 7f6cdc357640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6cd40b5270 0x7f6cd4152e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:49.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.445+0000 7f6cdc357640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cd4141d60 con 0x7f6cd406d2d0 2024-03-18T15:33:49.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.445+0000 7f6cdc357640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cd406d6b0 con 0x7f6cd40ae0f0 2024-03-18T15:33:49.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.445+0000 7f6cdc357640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f6cd40724e0 con 0x7f6cd40b5270 2024-03-18T15:33:49.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.445+0000 7f6cda8f4640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6cd40b5270 0x7f6cd4152e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:49.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.445+0000 7f6cda0f3640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f6cd406d2d0 0x7f6cd4141650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:57310/0 (socket says 172.21.15.5:57310) 2024-03-18T15:33:49.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.445+0000 7f6cda0f3640 1 -- 172.21.15.5:0/2294998528 learned_addr learned my addr 172.21.15.5:0/2294998528 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:33:49.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.445+0000 7f6cda8f4640 1 -- 172.21.15.5:0/2294998528 >> v1:172.21.15.80:6789/0 conn(0x7f6cd40ae0f0 legacy=0x7f6cd40b4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:49.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.449+0000 7f6cda8f4640 1 -- 172.21.15.5:0/2294998528 >> v1:172.21.15.5:6789/0 conn(0x7f6cd406d2d0 legacy=0x7f6cd4141650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:49.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.449+0000 7f6cda8f4640 1 -- 172.21.15.5:0/2294998528 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cd4072730 con 0x7f6cd40b5270 2024-03-18T15:33:49.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.449+0000 7f6cda8f4640 1 --2- 172.21.15.5:0/2294998528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cd40b5270 0x7f6cd4152e10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc009d90 tx=0x7f6ccc01c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaf5a23e9d099e1e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:49.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.449+0000 7f6cd90f1640 1 -- 172.21.15.5:0/2294998528 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc0a66c0 con 0x7f6cd40b5270 2024-03-18T15:33:49.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.449+0000 7f6cd90f1640 1 -- 172.21.15.5:0/2294998528 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ccc0a6860 con 0x7f6cd40b5270 2024-03-18T15:33:49.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.449+0000 7f6cd90f1640 1 -- 172.21.15.5:0/2294998528 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc0a6c30 con 0x7f6cd40b5270 2024-03-18T15:33:49.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.449+0000 7f6cdc357640 1 -- 172.21.15.5:0/2294998528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cd40b5270 msgr2=0x7f6cd4152e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:49.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.449+0000 7f6cdc357640 1 --2- 172.21.15.5:0/2294998528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cd40b5270 0x7f6cd4152e10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc009d90 tx=0x7f6ccc01c810 comp rx=0 tx=0).stop 2024-03-18T15:33:49.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.449+0000 7f6cdc357640 1 -- 172.21.15.5:0/2294998528 shutdown_connections 2024-03-18T15:33:49.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.449+0000 7f6cdc357640 1 --2- 172.21.15.5:0/2294998528 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cd40b5270 0x7f6cd4152e10 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.449+0000 7f6cdc357640 1 -- 172.21.15.5:0/2294998528 >> 172.21.15.5:0/2294998528 conn(0x7f6cd40b5670 msgr2=0x7f6cd40b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:49.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.449+0000 7f6cdc357640 1 -- 172.21.15.5:0/2294998528 shutdown_connections 2024-03-18T15:33:49.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.449+0000 7f6cdc357640 1 -- 172.21.15.5:0/2294998528 wait complete. 2024-03-18T15:33:49.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.449+0000 7f6cdc357640 1 Processor -- start 2024-03-18T15:33:49.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.449+0000 7f6cdc357640 1 -- start start 2024-03-18T15:33:49.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.453+0000 7f6cdc357640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6cd406d2d0 0x7f6cd41396f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:49.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.453+0000 7f6cdc357640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cd40ae0f0 0x7f6cd40b77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:49.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.453+0000 7f6cdc357640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6cd40b5270 0x7f6cd40b7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:49.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.453+0000 7f6cdc357640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd4155f40 con 0x7f6cd40ae0f0 2024-03-18T15:33:49.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.453+0000 7f6cdc357640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd4155dc0 con 0x7f6cd40b5270 2024-03-18T15:33:49.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.453+0000 7f6cdc357640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f6cd41560c0 con 0x7f6cd406d2d0 2024-03-18T15:33:49.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.453+0000 7f6cd98f2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cd40ae0f0 0x7f6cd40b77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:49.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.453+0000 7f6cd98f2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cd40ae0f0 0x7f6cd40b77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38624/0 (socket says 172.21.15.5:38624) 2024-03-18T15:33:49.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.453+0000 7f6cd98f2640 1 -- 172.21.15.5:0/1450316704 learned_addr learned my addr 172.21.15.5:0/1450316704 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:49.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.453+0000 7f6cda8f4640 1 --2- 172.21.15.5:0/1450316704 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6cd40b5270 0x7f6cd40b7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:49.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.453+0000 7f6cda0f3640 1 --2- 172.21.15.5:0/1450316704 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6cd406d2d0 0x7f6cd41396f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:49.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.453+0000 7f6cd98f2640 1 -- 172.21.15.5:0/1450316704 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6cd406d2d0 msgr2=0x7f6cd41396f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:49.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.453+0000 7f6cd98f2640 1 --2- 172.21.15.5:0/1450316704 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6cd406d2d0 0x7f6cd41396f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.453+0000 7f6cd98f2640 1 -- 172.21.15.5:0/1450316704 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6cd40b5270 msgr2=0x7f6cd40b7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:49.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.453+0000 7f6cd98f2640 1 --2- 172.21.15.5:0/1450316704 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6cd40b5270 0x7f6cd40b7cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.453+0000 7f6cd98f2640 1 -- 172.21.15.5:0/1450316704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cd40b6d60 con 0x7f6cd40ae0f0 2024-03-18T15:33:49.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.453+0000 7f6cd98f2640 1 --2- 172.21.15.5:0/1450316704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cd40ae0f0 0x7f6cd40b77b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6cc400c410 tx=0x7f6cc400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:49.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.453+0000 7f6ccaffd640 1 -- 172.21.15.5:0/1450316704 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4011790 con 0x7f6cd40ae0f0 2024-03-18T15:33:49.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.453+0000 7f6ccaffd640 1 -- 172.21.15.5:0/1450316704 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cc4017070 con 0x7f6cd40ae0f0 2024-03-18T15:33:49.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.453+0000 7f6cdc357640 1 -- 172.21.15.5:0/1450316704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cd4136330 con 0x7f6cd40ae0f0 2024-03-18T15:33:49.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.453+0000 7f6ccaffd640 1 -- 172.21.15.5:0/1450316704 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4011bb0 con 0x7f6cd40ae0f0 2024-03-18T15:33:49.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.453+0000 7f6cdc357640 1 -- 172.21.15.5:0/1450316704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cd40b6fd0 con 0x7f6cd40ae0f0 2024-03-18T15:33:49.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.453+0000 7f6cdc357640 1 -- 172.21.15.5:0/1450316704 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cd413de50 con 0x7f6cd40ae0f0 2024-03-18T15:33:49.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.453+0000 7f6ccaffd640 1 -- 172.21.15.5:0/1450316704 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f6cc401c460 con 0x7f6cd40ae0f0 2024-03-18T15:33:49.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.453+0000 7f6ccaffd640 1 --2- 172.21.15.5:0/1450316704 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6cb8067690 0x7f6cb8069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:49.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.453+0000 7f6ccaffd640 1 -- 172.21.15.5:0/1450316704 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f6cc40854d0 con 0x7f6cd40ae0f0 2024-03-18T15:33:49.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.453+0000 7f6cda0f3640 1 --2- 172.21.15.5:0/1450316704 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6cb8067690 0x7f6cb8069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:49.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.453+0000 7f6cda0f3640 1 --2- 172.21.15.5:0/1450316704 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6cb8067690 0x7f6cb8069b50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6cd41454a0 tx=0x7f6cd0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:49.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.457+0000 7f6ccaffd640 1 -- 172.21.15.5:0/1450316704 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f6cc4021020 con 0x7f6cd40ae0f0 2024-03-18T15:33:49.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.609+0000 7f6cdc357640 1 -- 172.21.15.5:0/1450316704 --> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6cd404f550 con 0x7f6cb8067690 2024-03-18T15:33:49.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.617+0000 7f6ccaffd640 1 -- 172.21.15.5:0/1450316704 <== mgr.4110 v2:172.21.15.5:6832/3362034860 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251932 (secure 0 0 0) 0x7f6cd404f550 con 0x7f6cb8067690 2024-03-18T15:33:49.621 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:33:49.622 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-03-18T15:33:49.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.621+0000 7f6cdc357640 1 -- 172.21.15.5:0/1450316704 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6cb8067690 msgr2=0x7f6cb8069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:49.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.621+0000 7f6cdc357640 1 --2- 172.21.15.5:0/1450316704 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6cb8067690 0x7f6cb8069b50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6cd41454a0 tx=0x7f6cd0009040 comp rx=0 tx=0).stop 2024-03-18T15:33:49.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.621+0000 7f6cdc357640 1 -- 172.21.15.5:0/1450316704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cd40ae0f0 msgr2=0x7f6cd40b77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:49.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.621+0000 7f6cdc357640 1 --2- 172.21.15.5:0/1450316704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cd40ae0f0 0x7f6cd40b77b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6cc400c410 tx=0x7f6cc400c8d0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.621+0000 7f6cdc357640 1 -- 172.21.15.5:0/1450316704 shutdown_connections 2024-03-18T15:33:49.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.621+0000 7f6cdc357640 1 --2- 172.21.15.5:0/1450316704 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6cb8067690 0x7f6cb8069b50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.621+0000 7f6cdc357640 1 --2- 172.21.15.5:0/1450316704 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6cd40b5270 0x7f6cd40b7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.621+0000 7f6cdc357640 1 --2- 172.21.15.5:0/1450316704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6cd40ae0f0 0x7f6cd40b77b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.621+0000 7f6cdc357640 1 --2- 172.21.15.5:0/1450316704 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6cd406d2d0 0x7f6cd41396f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.621+0000 7f6cdc357640 1 -- 172.21.15.5:0/1450316704 >> 172.21.15.5:0/1450316704 conn(0x7f6cd40b5670 msgr2=0x7f6cd406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:49.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.621+0000 7f6cdc357640 1 -- 172.21.15.5:0/1450316704 shutdown_connections 2024-03-18T15:33:49.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.621+0000 7f6cdc357640 1 -- 172.21.15.5:0/1450316704 wait complete. 2024-03-18T15:33:49.639 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-03-18T15:33:48.887212+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":248160,"kb_used_data":2856,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754726560,"statfs":{"total":773094113280,"available":772839997440,"internally_reserved":0,"allocated":2924544,"data_stored":1773196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.922001"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.994364+0000","last_change":"2024-03-18T15:33:42.998484+0000","last_active":"2024-03-18T15:33:43.994364+0000","last_peered":"2024-03-18T15:33:43.994364+0000","last_clean":"2024-03-18T15:33:43.994364+0000","last_became_active":"2024-03-18T15:33:42.996054+0000","last_became_peered":"2024-03-18T15:33:42.996054+0000","last_unstale":"2024-03-18T15:33:43.994364+0000","last_undegraded":"2024-03-18T15:33:43.994364+0000","last_fullsized":"2024-03-18T15:33:43.994364+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:38:03.814592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.996595+0000","last_change":"2024-03-18T15:33:42.997522+0000","last_active":"2024-03-18T15:33:43.996595+0000","last_peered":"2024-03-18T15:33:43.996595+0000","last_clean":"2024-03-18T15:33:43.996595+0000","last_became_active":"2024-03-18T15:33:42.997164+0000","last_became_peered":"2024-03-18T15:33:42.997164+0000","last_unstale":"2024-03-18T15:33:43.996595+0000","last_undegraded":"2024-03-18T15:33:43.996595+0000","last_fullsized":"2024-03-18T15:33:43.996595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:29:18.287234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.096933+0000","last_change":"2024-03-18T15:33:43.001309+0000","last_active":"2024-03-18T15:33:45.096933+0000","last_peered":"2024-03-18T15:33:45.096933+0000","last_clean":"2024-03-18T15:33:45.096933+0000","last_became_active":"2024-03-18T15:33:43.000919+0000","last_became_peered":"2024-03-18T15:33:43.000919+0000","last_unstale":"2024-03-18T15:33:45.096933+0000","last_undegraded":"2024-03-18T15:33:45.096933+0000","last_fullsized":"2024-03-18T15:33:45.096933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:10:31.661870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.095996+0000","last_change":"2024-03-18T15:33:43.022575+0000","last_active":"2024-03-18T15:33:45.095996+0000","last_peered":"2024-03-18T15:33:45.095996+0000","last_clean":"2024-03-18T15:33:45.095996+0000","last_became_active":"2024-03-18T15:33:43.022221+0000","last_became_peered":"2024-03-18T15:33:43.022221+0000","last_unstale":"2024-03-18T15:33:45.095996+0000","last_undegraded":"2024-03-18T15:33:45.095996+0000","last_fullsized":"2024-03-18T15:33:45.095996+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:59:03.138076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.994821+0000","last_change":"2024-03-18T15:33:42.999245+0000","last_active":"2024-03-18T15:33:43.994821+0000","last_peered":"2024-03-18T15:33:43.994821+0000","last_clean":"2024-03-18T15:33:43.994821+0000","last_became_active":"2024-03-18T15:33:42.998951+0000","last_became_peered":"2024-03-18T15:33:42.998951+0000","last_unstale":"2024-03-18T15:33:43.994821+0000","last_undegraded":"2024-03-18T15:33:43.994821+0000","last_fullsized":"2024-03-18T15:33:43.994821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:51:36.500416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.095710+0000","last_change":"2024-03-18T15:33:43.017400+0000","last_active":"2024-03-18T15:33:45.095710+0000","last_peered":"2024-03-18T15:33:45.095710+0000","last_clean":"2024-03-18T15:33:45.095710+0000","last_became_active":"2024-03-18T15:33:43.017074+0000","last_became_peered":"2024-03-18T15:33:43.017074+0000","last_unstale":"2024-03-18T15:33:45.095710+0000","last_undegraded":"2024-03-18T15:33:45.095710+0000","last_fullsized":"2024-03-18T15:33:45.095710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:50:28.968807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.993424+0000","last_change":"2024-03-18T15:33:42.998214+0000","last_active":"2024-03-18T15:33:43.993424+0000","last_peered":"2024-03-18T15:33:43.993424+0000","last_clean":"2024-03-18T15:33:43.993424+0000","last_became_active":"2024-03-18T15:33:42.997922+0000","last_became_peered":"2024-03-18T15:33:42.997922+0000","last_unstale":"2024-03-18T15:33:43.993424+0000","last_undegraded":"2024-03-18T15:33:43.993424+0000","last_fullsized":"2024-03-18T15:33:43.993424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:53:24.132151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.096323+0000","last_change":"2024-03-18T15:33:43.021183+0000","last_active":"2024-03-18T15:33:45.096323+0000","last_peered":"2024-03-18T15:33:45.096323+0000","last_clean":"2024-03-18T15:33:45.096323+0000","last_became_active":"2024-03-18T15:33:43.020848+0000","last_became_peered":"2024-03-18T15:33:43.020848+0000","last_unstale":"2024-03-18T15:33:45.096323+0000","last_undegraded":"2024-03-18T15:33:45.096323+0000","last_fullsized":"2024-03-18T15:33:45.096323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:06:40.477304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.994576+0000","last_change":"2024-03-18T15:33:43.018236+0000","last_active":"2024-03-18T15:33:43.994576+0000","last_peered":"2024-03-18T15:33:43.994576+0000","last_clean":"2024-03-18T15:33:43.994576+0000","last_became_active":"2024-03-18T15:33:43.017781+0000","last_became_peered":"2024-03-18T15:33:43.017781+0000","last_unstale":"2024-03-18T15:33:43.994576+0000","last_undegraded":"2024-03-18T15:33:43.994576+0000","last_fullsized":"2024-03-18T15:33:43.994576+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:57:09.423392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.002529+0000","last_change":"2024-03-18T15:33:43.020001+0000","last_active":"2024-03-18T15:33:44.002529+0000","last_peered":"2024-03-18T15:33:44.002529+0000","last_clean":"2024-03-18T15:33:44.002529+0000","last_became_active":"2024-03-18T15:33:43.019640+0000","last_became_peered":"2024-03-18T15:33:43.019640+0000","last_unstale":"2024-03-18T15:33:44.002529+0000","last_undegraded":"2024-03-18T15:33:44.002529+0000","last_fullsized":"2024-03-18T15:33:44.002529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:28:55.955807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.002450+0000","last_change":"2024-03-18T15:33:43.020077+0000","last_active":"2024-03-18T15:33:44.002450+0000","last_peered":"2024-03-18T15:33:44.002450+0000","last_clean":"2024-03-18T15:33:44.002450+0000","last_became_active":"2024-03-18T15:33:43.019703+0000","last_became_peered":"2024-03-18T15:33:43.019703+0000","last_unstale":"2024-03-18T15:33:44.002450+0000","last_undegraded":"2024-03-18T15:33:44.002450+0000","last_fullsized":"2024-03-18T15:33:44.002450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:10:31.661870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.994862+0000","last_change":"2024-03-18T15:33:42.988744+0000","last_active":"2024-03-18T15:33:43.994862+0000","last_peered":"2024-03-18T15:33:43.994862+0000","last_clean":"2024-03-18T15:33:43.994862+0000","last_became_active":"2024-03-18T15:33:42.988355+0000","last_became_peered":"2024-03-18T15:33:42.988355+0000","last_unstale":"2024-03-18T15:33:43.994862+0000","last_undegraded":"2024-03-18T15:33:43.994862+0000","last_fullsized":"2024-03-18T15:33:43.994862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:43:42.420536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.098025+0000","last_change":"2024-03-18T15:33:43.010027+0000","last_active":"2024-03-18T15:33:45.098025+0000","last_peered":"2024-03-18T15:33:45.098025+0000","last_clean":"2024-03-18T15:33:45.098025+0000","last_became_active":"2024-03-18T15:33:43.009401+0000","last_became_peered":"2024-03-18T15:33:43.009401+0000","last_unstale":"2024-03-18T15:33:45.098025+0000","last_undegraded":"2024-03-18T15:33:45.098025+0000","last_fullsized":"2024-03-18T15:33:45.098025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:11:30.652938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.096064+0000","last_change":"2024-03-18T15:33:42.999478+0000","last_active":"2024-03-18T15:33:45.096064+0000","last_peered":"2024-03-18T15:33:45.096064+0000","last_clean":"2024-03-18T15:33:45.096064+0000","last_became_active":"2024-03-18T15:33:42.998918+0000","last_became_peered":"2024-03-18T15:33:42.998918+0000","last_unstale":"2024-03-18T15:33:45.096064+0000","last_undegraded":"2024-03-18T15:33:45.096064+0000","last_fullsized":"2024-03-18T15:33:45.096064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:48:33.731710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.995511+0000","last_change":"2024-03-18T15:33:42.983262+0000","last_active":"2024-03-18T15:33:43.995511+0000","last_peered":"2024-03-18T15:33:43.995511+0000","last_clean":"2024-03-18T15:33:43.995511+0000","last_became_active":"2024-03-18T15:33:42.982356+0000","last_became_peered":"2024-03-18T15:33:42.982356+0000","last_unstale":"2024-03-18T15:33:43.995511+0000","last_undegraded":"2024-03-18T15:33:43.995511+0000","last_fullsized":"2024-03-18T15:33:43.995511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:16:44.575392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.095621+0000","last_change":"2024-03-18T15:33:43.017589+0000","last_active":"2024-03-18T15:33:45.095621+0000","last_peered":"2024-03-18T15:33:45.095621+0000","last_clean":"2024-03-18T15:33:45.095621+0000","last_became_active":"2024-03-18T15:33:43.016991+0000","last_became_peered":"2024-03-18T15:33:43.016991+0000","last_unstale":"2024-03-18T15:33:45.095621+0000","last_undegraded":"2024-03-18T15:33:45.095621+0000","last_fullsized":"2024-03-18T15:33:45.095621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:37:33.748962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.096964+0000","last_change":"2024-03-18T15:33:43.015021+0000","last_active":"2024-03-18T15:33:45.096964+0000","last_peered":"2024-03-18T15:33:45.096964+0000","last_clean":"2024-03-18T15:33:45.096964+0000","last_became_active":"2024-03-18T15:33:43.012689+0000","last_became_peered":"2024-03-18T15:33:43.012689+0000","last_unstale":"2024-03-18T15:33:45.096964+0000","last_undegraded":"2024-03-18T15:33:45.096964+0000","last_fullsized":"2024-03-18T15:33:45.096964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:19:40.976122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.869526+0000","last_change":"2024-03-18T15:33:42.997254+0000","last_active":"2024-03-18T15:33:44.869526+0000","last_peered":"2024-03-18T15:33:44.869526+0000","last_clean":"2024-03-18T15:33:44.869526+0000","last_became_active":"2024-03-18T15:33:42.995704+0000","last_became_peered":"2024-03-18T15:33:42.995704+0000","last_unstale":"2024-03-18T15:33:44.869526+0000","last_undegraded":"2024-03-18T15:33:44.869526+0000","last_fullsized":"2024-03-18T15:33:44.869526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:13:27.226842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.994025+0000","last_change":"2024-03-18T15:33:42.998872+0000","last_active":"2024-03-18T15:33:43.994025+0000","last_peered":"2024-03-18T15:33:43.994025+0000","last_clean":"2024-03-18T15:33:43.994025+0000","last_became_active":"2024-03-18T15:33:42.998529+0000","last_became_peered":"2024-03-18T15:33:42.998529+0000","last_unstale":"2024-03-18T15:33:43.994025+0000","last_undegraded":"2024-03-18T15:33:43.994025+0000","last_fullsized":"2024-03-18T15:33:43.994025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:00:41.383860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.002213+0000","last_change":"2024-03-18T15:33:43.003843+0000","last_active":"2024-03-18T15:33:44.002213+0000","last_peered":"2024-03-18T15:33:44.002213+0000","last_clean":"2024-03-18T15:33:44.002213+0000","last_became_active":"2024-03-18T15:33:43.003481+0000","last_became_peered":"2024-03-18T15:33:43.003481+0000","last_unstale":"2024-03-18T15:33:44.002213+0000","last_undegraded":"2024-03-18T15:33:44.002213+0000","last_fullsized":"2024-03-18T15:33:44.002213+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:21:20.566649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.096775+0000","last_change":"2024-03-18T15:33:43.016217+0000","last_active":"2024-03-18T15:33:45.096775+0000","last_peered":"2024-03-18T15:33:45.096775+0000","last_clean":"2024-03-18T15:33:45.096775+0000","last_became_active":"2024-03-18T15:33:43.015511+0000","last_became_peered":"2024-03-18T15:33:43.015511+0000","last_unstale":"2024-03-18T15:33:45.096775+0000","last_undegraded":"2024-03-18T15:33:45.096775+0000","last_fullsized":"2024-03-18T15:33:45.096775+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:50:43.021257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.095772+0000","last_change":"2024-03-18T15:33:43.014558+0000","last_active":"2024-03-18T15:33:45.095772+0000","last_peered":"2024-03-18T15:33:45.095772+0000","last_clean":"2024-03-18T15:33:45.095772+0000","last_became_active":"2024-03-18T15:33:43.012745+0000","last_became_peered":"2024-03-18T15:33:43.012745+0000","last_unstale":"2024-03-18T15:33:45.095772+0000","last_undegraded":"2024-03-18T15:33:45.095772+0000","last_fullsized":"2024-03-18T15:33:45.095772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:24:37.779979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.097432+0000","last_change":"2024-03-18T15:33:43.010632+0000","last_active":"2024-03-18T15:33:45.097432+0000","last_peered":"2024-03-18T15:33:45.097432+0000","last_clean":"2024-03-18T15:33:45.097432+0000","last_became_active":"2024-03-18T15:33:43.010198+0000","last_became_peered":"2024-03-18T15:33:43.010198+0000","last_unstale":"2024-03-18T15:33:45.097432+0000","last_undegraded":"2024-03-18T15:33:45.097432+0000","last_fullsized":"2024-03-18T15:33:45.097432+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:03:19.737667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.095654+0000","last_change":"2024-03-18T15:33:43.018016+0000","last_active":"2024-03-18T15:33:45.095654+0000","last_peered":"2024-03-18T15:33:45.095654+0000","last_clean":"2024-03-18T15:33:45.095654+0000","last_became_active":"2024-03-18T15:33:43.017655+0000","last_became_peered":"2024-03-18T15:33:43.017655+0000","last_unstale":"2024-03-18T15:33:45.095654+0000","last_undegraded":"2024-03-18T15:33:45.095654+0000","last_fullsized":"2024-03-18T15:33:45.095654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T15:41:33.909895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.099107+0000","last_change":"2024-03-18T15:33:43.010776+0000","last_active":"2024-03-18T15:33:45.099107+0000","last_peered":"2024-03-18T15:33:45.099107+0000","last_clean":"2024-03-18T15:33:45.099107+0000","last_became_active":"2024-03-18T15:33:43.010222+0000","last_became_peered":"2024-03-18T15:33:43.010222+0000","last_unstale":"2024-03-18T15:33:45.099107+0000","last_undegraded":"2024-03-18T15:33:45.099107+0000","last_fullsized":"2024-03-18T15:33:45.099107+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:53:24.132151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.002604+0000","last_change":"2024-03-18T15:33:43.011621+0000","last_active":"2024-03-18T15:33:44.002604+0000","last_peered":"2024-03-18T15:33:44.002604+0000","last_clean":"2024-03-18T15:33:44.002604+0000","last_became_active":"2024-03-18T15:33:43.011354+0000","last_became_peered":"2024-03-18T15:33:43.011354+0000","last_unstale":"2024-03-18T15:33:44.002604+0000","last_undegraded":"2024-03-18T15:33:44.002604+0000","last_fullsized":"2024-03-18T15:33:44.002604+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:50:28.968807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.996192+0000","last_change":"2024-03-18T15:33:43.011713+0000","last_active":"2024-03-18T15:33:43.996192+0000","last_peered":"2024-03-18T15:33:43.996192+0000","last_clean":"2024-03-18T15:33:43.996192+0000","last_became_active":"2024-03-18T15:33:43.011288+0000","last_became_peered":"2024-03-18T15:33:43.011288+0000","last_unstale":"2024-03-18T15:33:43.996192+0000","last_undegraded":"2024-03-18T15:33:43.996192+0000","last_fullsized":"2024-03-18T15:33:43.996192+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:07:37.987066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.098603+0000","last_change":"2024-03-18T15:33:43.018468+0000","last_active":"2024-03-18T15:33:45.098603+0000","last_peered":"2024-03-18T15:33:45.098603+0000","last_clean":"2024-03-18T15:33:45.098603+0000","last_became_active":"2024-03-18T15:33:43.012834+0000","last_became_peered":"2024-03-18T15:33:43.012834+0000","last_unstale":"2024-03-18T15:33:45.098603+0000","last_undegraded":"2024-03-18T15:33:45.098603+0000","last_fullsized":"2024-03-18T15:33:45.098603+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:07:37.987066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.096707+0000","last_change":"2024-03-18T15:33:43.016590+0000","last_active":"2024-03-18T15:33:45.096707+0000","last_peered":"2024-03-18T15:33:45.096707+0000","last_clean":"2024-03-18T15:33:45.096707+0000","last_became_active":"2024-03-18T15:33:43.015944+0000","last_became_peered":"2024-03-18T15:33:43.015944+0000","last_unstale":"2024-03-18T15:33:45.096707+0000","last_undegraded":"2024-03-18T15:33:45.096707+0000","last_fullsized":"2024-03-18T15:33:45.096707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:57:58.192115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.096097+0000","last_change":"2024-03-18T15:33:43.014278+0000","last_active":"2024-03-18T15:33:45.096097+0000","last_peered":"2024-03-18T15:33:45.096097+0000","last_clean":"2024-03-18T15:33:45.096097+0000","last_became_active":"2024-03-18T15:33:43.012307+0000","last_became_peered":"2024-03-18T15:33:43.012307+0000","last_unstale":"2024-03-18T15:33:45.096097+0000","last_undegraded":"2024-03-18T15:33:45.096097+0000","last_fullsized":"2024-03-18T15:33:45.096097+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:54:41.008899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.869746+0000","last_change":"2024-03-18T15:33:43.018853+0000","last_active":"2024-03-18T15:33:44.869746+0000","last_peered":"2024-03-18T15:33:44.869746+0000","last_clean":"2024-03-18T15:33:44.869746+0000","last_became_active":"2024-03-18T15:33:43.018496+0000","last_became_peered":"2024-03-18T15:33:43.018496+0000","last_unstale":"2024-03-18T15:33:44.869746+0000","last_undegraded":"2024-03-18T15:33:44.869746+0000","last_fullsized":"2024-03-18T15:33:44.869746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:43:00.095618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.002017+0000","last_change":"2024-03-18T15:33:43.000804+0000","last_active":"2024-03-18T15:33:44.002017+0000","last_peered":"2024-03-18T15:33:44.002017+0000","last_clean":"2024-03-18T15:33:44.002017+0000","last_became_active":"2024-03-18T15:33:43.000463+0000","last_became_peered":"2024-03-18T15:33:43.000463+0000","last_unstale":"2024-03-18T15:33:44.002017+0000","last_undegraded":"2024-03-18T15:33:44.002017+0000","last_fullsized":"2024-03-18T15:33:44.002017+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:57:58.192115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.995711+0000","last_change":"2024-03-18T15:33:42.995621+0000","last_active":"2024-03-18T15:33:43.995711+0000","last_peered":"2024-03-18T15:33:43.995711+0000","last_clean":"2024-03-18T15:33:43.995711+0000","last_became_active":"2024-03-18T15:33:42.995289+0000","last_became_peered":"2024-03-18T15:33:42.995289+0000","last_unstale":"2024-03-18T15:33:43.995711+0000","last_undegraded":"2024-03-18T15:33:43.995711+0000","last_fullsized":"2024-03-18T15:33:43.995711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:07:05.101032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.098186+0000","last_change":"2024-03-18T15:33:42.010459+0000","last_active":"2024-03-18T15:33:45.098186+0000","last_peered":"2024-03-18T15:33:45.098186+0000","last_clean":"2024-03-18T15:33:45.098186+0000","last_became_active":"2024-03-18T15:33:42.009978+0000","last_became_peered":"2024-03-18T15:33:42.009978+0000","last_unstale":"2024-03-18T15:33:45.098186+0000","last_undegraded":"2024-03-18T15:33:45.098186+0000","last_fullsized":"2024-03-18T15:33:45.098186+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:20:19.179099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.096910+0000","last_change":"2024-03-18T15:33:43.015925+0000","last_active":"2024-03-18T15:33:45.096910+0000","last_peered":"2024-03-18T15:33:45.096910+0000","last_clean":"2024-03-18T15:33:45.096910+0000","last_became_active":"2024-03-18T15:33:43.014716+0000","last_became_peered":"2024-03-18T15:33:43.014716+0000","last_unstale":"2024-03-18T15:33:45.096910+0000","last_undegraded":"2024-03-18T15:33:45.096910+0000","last_fullsized":"2024-03-18T15:33:45.096910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:21:20.566649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.002101+0000","last_change":"2024-03-18T15:33:42.012969+0000","last_active":"2024-03-18T15:33:44.002101+0000","last_peered":"2024-03-18T15:33:44.002101+0000","last_clean":"2024-03-18T15:33:44.002101+0000","last_became_active":"2024-03-18T15:33:42.012416+0000","last_became_peered":"2024-03-18T15:33:42.012416+0000","last_unstale":"2024-03-18T15:33:44.002101+0000","last_undegraded":"2024-03-18T15:33:44.002101+0000","last_fullsized":"2024-03-18T15:33:44.002101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:47:53.438385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.097620+0000","last_change":"2024-03-18T15:33:41.996706+0000","last_active":"2024-03-18T15:33:45.097620+0000","last_peered":"2024-03-18T15:33:45.097620+0000","last_clean":"2024-03-18T15:33:45.097620+0000","last_became_active":"2024-03-18T15:33:41.996414+0000","last_became_peered":"2024-03-18T15:33:41.996414+0000","last_unstale":"2024-03-18T15:33:45.097620+0000","last_undegraded":"2024-03-18T15:33:45.097620+0000","last_fullsized":"2024-03-18T15:33:45.097620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:48:24.038250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.096753+0000","last_change":"2024-03-18T15:33:43.009392+0000","last_active":"2024-03-18T15:33:45.096753+0000","last_peered":"2024-03-18T15:33:45.096753+0000","last_clean":"2024-03-18T15:33:45.096753+0000","last_became_active":"2024-03-18T15:33:42.997447+0000","last_became_peered":"2024-03-18T15:33:42.997447+0000","last_unstale":"2024-03-18T15:33:45.096753+0000","last_undegraded":"2024-03-18T15:33:45.096753+0000","last_fullsized":"2024-03-18T15:33:45.096753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:35:43.006937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.096207+0000","last_change":"2024-03-18T15:33:42.002322+0000","last_active":"2024-03-18T15:33:45.096207+0000","last_peered":"2024-03-18T15:33:45.096207+0000","last_clean":"2024-03-18T15:33:45.096207+0000","last_became_active":"2024-03-18T15:33:42.001646+0000","last_became_peered":"2024-03-18T15:33:42.001646+0000","last_unstale":"2024-03-18T15:33:45.096207+0000","last_undegraded":"2024-03-18T15:33:45.096207+0000","last_fullsized":"2024-03-18T15:33:45.096207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:39:46.475100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.995243+0000","last_change":"2024-03-18T15:33:42.998180+0000","last_active":"2024-03-18T15:33:43.995243+0000","last_peered":"2024-03-18T15:33:43.995243+0000","last_clean":"2024-03-18T15:33:43.995243+0000","last_became_active":"2024-03-18T15:33:42.997429+0000","last_became_peered":"2024-03-18T15:33:42.997429+0000","last_unstale":"2024-03-18T15:33:43.995243+0000","last_undegraded":"2024-03-18T15:33:43.995243+0000","last_fullsized":"2024-03-18T15:33:43.995243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:03:19.737667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.002488+0000","last_change":"2024-03-18T15:33:42.005707+0000","last_active":"2024-03-18T15:33:44.002488+0000","last_peered":"2024-03-18T15:33:44.002488+0000","last_clean":"2024-03-18T15:33:44.002488+0000","last_became_active":"2024-03-18T15:33:42.004925+0000","last_became_peered":"2024-03-18T15:33:42.004925+0000","last_unstale":"2024-03-18T15:33:44.002488+0000","last_undegraded":"2024-03-18T15:33:44.002488+0000","last_fullsized":"2024-03-18T15:33:44.002488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:39:46.475100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.096274+0000","last_change":"2024-03-18T15:33:43.012260+0000","last_active":"2024-03-18T15:33:45.096274+0000","last_peered":"2024-03-18T15:33:45.096274+0000","last_clean":"2024-03-18T15:33:45.096274+0000","last_became_active":"2024-03-18T15:33:43.011884+0000","last_became_peered":"2024-03-18T15:33:43.011884+0000","last_unstale":"2024-03-18T15:33:45.096274+0000","last_undegraded":"2024-03-18T15:33:45.096274+0000","last_fullsized":"2024-03-18T15:33:45.096274+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:33:26.178317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.096423+0000","last_change":"2024-03-18T15:33:42.006507+0000","last_active":"2024-03-18T15:33:45.096423+0000","last_peered":"2024-03-18T15:33:45.096423+0000","last_clean":"2024-03-18T15:33:45.096423+0000","last_became_active":"2024-03-18T15:33:42.005178+0000","last_became_peered":"2024-03-18T15:33:42.005178+0000","last_unstale":"2024-03-18T15:33:45.096423+0000","last_undegraded":"2024-03-18T15:33:45.096423+0000","last_fullsized":"2024-03-18T15:33:45.096423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:09:14.869832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.097557+0000","last_change":"2024-03-18T15:33:43.021210+0000","last_active":"2024-03-18T15:33:45.097557+0000","last_peered":"2024-03-18T15:33:45.097557+0000","last_clean":"2024-03-18T15:33:45.097557+0000","last_became_active":"2024-03-18T15:33:43.020819+0000","last_became_peered":"2024-03-18T15:33:43.020819+0000","last_unstale":"2024-03-18T15:33:45.097557+0000","last_undegraded":"2024-03-18T15:33:45.097557+0000","last_fullsized":"2024-03-18T15:33:45.097557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:50:28.968807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.996129+0000","last_change":"2024-03-18T15:33:42.000763+0000","last_active":"2024-03-18T15:33:43.996129+0000","last_peered":"2024-03-18T15:33:43.996129+0000","last_clean":"2024-03-18T15:33:43.996129+0000","last_became_active":"2024-03-18T15:33:42.000392+0000","last_became_peered":"2024-03-18T15:33:42.000392+0000","last_unstale":"2024-03-18T15:33:43.996129+0000","last_undegraded":"2024-03-18T15:33:43.996129+0000","last_fullsized":"2024-03-18T15:33:43.996129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:43:41.408584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.995096+0000","last_change":"2024-03-18T15:33:43.003710+0000","last_active":"2024-03-18T15:33:43.995096+0000","last_peered":"2024-03-18T15:33:43.995096+0000","last_clean":"2024-03-18T15:33:43.995096+0000","last_became_active":"2024-03-18T15:33:43.003388+0000","last_became_peered":"2024-03-18T15:33:43.003388+0000","last_unstale":"2024-03-18T15:33:43.995096+0000","last_undegraded":"2024-03-18T15:33:43.995096+0000","last_fullsized":"2024-03-18T15:33:43.995096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:33:12.492530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.002393+0000","last_change":"2024-03-18T15:33:41.993113+0000","last_active":"2024-03-18T15:33:44.002393+0000","last_peered":"2024-03-18T15:33:44.002393+0000","last_clean":"2024-03-18T15:33:44.002393+0000","last_became_active":"2024-03-18T15:33:41.992459+0000","last_became_peered":"2024-03-18T15:33:41.992459+0000","last_unstale":"2024-03-18T15:33:44.002393+0000","last_undegraded":"2024-03-18T15:33:44.002393+0000","last_fullsized":"2024-03-18T15:33:44.002393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:26:02.875362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.993666+0000","last_change":"2024-03-18T15:33:43.010684+0000","last_active":"2024-03-18T15:33:43.993666+0000","last_peered":"2024-03-18T15:33:43.993666+0000","last_clean":"2024-03-18T15:33:43.993666+0000","last_became_active":"2024-03-18T15:33:43.010021+0000","last_became_peered":"2024-03-18T15:33:43.010021+0000","last_unstale":"2024-03-18T15:33:43.993666+0000","last_undegraded":"2024-03-18T15:33:43.993666+0000","last_fullsized":"2024-03-18T15:33:43.993666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:19:11.577023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.096901+0000","last_change":"2024-03-18T15:33:41.991859+0000","last_active":"2024-03-18T15:33:45.096901+0000","last_peered":"2024-03-18T15:33:45.096901+0000","last_clean":"2024-03-18T15:33:45.096901+0000","last_became_active":"2024-03-18T15:33:41.987429+0000","last_became_peered":"2024-03-18T15:33:41.987429+0000","last_unstale":"2024-03-18T15:33:45.096901+0000","last_undegraded":"2024-03-18T15:33:45.096901+0000","last_fullsized":"2024-03-18T15:33:45.096901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:41:48.671691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.096587+0000","last_change":"2024-03-18T15:33:43.014886+0000","last_active":"2024-03-18T15:33:45.096587+0000","last_peered":"2024-03-18T15:33:45.096587+0000","last_clean":"2024-03-18T15:33:45.096587+0000","last_became_active":"2024-03-18T15:33:43.014430+0000","last_became_peered":"2024-03-18T15:33:43.014430+0000","last_unstale":"2024-03-18T15:33:45.096587+0000","last_undegraded":"2024-03-18T15:33:45.096587+0000","last_fullsized":"2024-03-18T15:33:45.096587+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:59:23.040679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.095989+0000","last_change":"2024-03-18T15:33:42.000527+0000","last_active":"2024-03-18T15:33:45.095989+0000","last_peered":"2024-03-18T15:33:45.095989+0000","last_clean":"2024-03-18T15:33:45.095989+0000","last_became_active":"2024-03-18T15:33:41.999663+0000","last_became_peered":"2024-03-18T15:33:41.999663+0000","last_unstale":"2024-03-18T15:33:45.095989+0000","last_undegraded":"2024-03-18T15:33:45.095989+0000","last_fullsized":"2024-03-18T15:33:45.095989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:26:02.875362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.994248+0000","last_change":"2024-03-18T15:33:42.997482+0000","last_active":"2024-03-18T15:33:43.994248+0000","last_peered":"2024-03-18T15:33:43.994248+0000","last_clean":"2024-03-18T15:33:43.994248+0000","last_became_active":"2024-03-18T15:33:42.993674+0000","last_became_peered":"2024-03-18T15:33:42.993674+0000","last_unstale":"2024-03-18T15:33:43.994248+0000","last_undegraded":"2024-03-18T15:33:43.994248+0000","last_fullsized":"2024-03-18T15:33:43.994248+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:04:58.261410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.001972+0000","last_change":"2024-03-18T15:33:41.990751+0000","last_active":"2024-03-18T15:33:44.001972+0000","last_peered":"2024-03-18T15:33:44.001972+0000","last_clean":"2024-03-18T15:33:44.001972+0000","last_became_active":"2024-03-18T15:33:41.989537+0000","last_became_peered":"2024-03-18T15:33:41.989537+0000","last_unstale":"2024-03-18T15:33:44.001972+0000","last_undegraded":"2024-03-18T15:33:44.001972+0000","last_fullsized":"2024-03-18T15:33:44.001972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:09:14.869832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.002581+0000","last_change":"2024-03-18T15:33:43.000356+0000","last_active":"2024-03-18T15:33:44.002581+0000","last_peered":"2024-03-18T15:33:44.002581+0000","last_clean":"2024-03-18T15:33:44.002581+0000","last_became_active":"2024-03-18T15:33:43.000031+0000","last_became_peered":"2024-03-18T15:33:43.000031+0000","last_unstale":"2024-03-18T15:33:44.002581+0000","last_undegraded":"2024-03-18T15:33:44.002581+0000","last_fullsized":"2024-03-18T15:33:44.002581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:24:37.779979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.869203+0000","last_change":"2024-03-18T15:33:41.976955+0000","last_active":"2024-03-18T15:33:44.869203+0000","last_peered":"2024-03-18T15:33:44.869203+0000","last_clean":"2024-03-18T15:33:44.869203+0000","last_became_active":"2024-03-18T15:33:41.976330+0000","last_became_peered":"2024-03-18T15:33:41.976330+0000","last_unstale":"2024-03-18T15:33:44.869203+0000","last_undegraded":"2024-03-18T15:33:44.869203+0000","last_fullsized":"2024-03-18T15:33:44.869203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:38:45.057586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.993542+0000","last_change":"2024-03-18T15:33:42.994227+0000","last_active":"2024-03-18T15:33:43.993542+0000","last_peered":"2024-03-18T15:33:43.993542+0000","last_clean":"2024-03-18T15:33:43.993542+0000","last_became_active":"2024-03-18T15:33:42.993870+0000","last_became_peered":"2024-03-18T15:33:42.993870+0000","last_unstale":"2024-03-18T15:33:43.993542+0000","last_undegraded":"2024-03-18T15:33:43.993542+0000","last_fullsized":"2024-03-18T15:33:43.993542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:29:15.496652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.994431+0000","last_change":"2024-03-18T15:33:41.987050+0000","last_active":"2024-03-18T15:33:43.994431+0000","last_peered":"2024-03-18T15:33:43.994431+0000","last_clean":"2024-03-18T15:33:43.994431+0000","last_became_active":"2024-03-18T15:33:41.986209+0000","last_became_peered":"2024-03-18T15:33:41.986209+0000","last_unstale":"2024-03-18T15:33:43.994431+0000","last_undegraded":"2024-03-18T15:33:43.994431+0000","last_fullsized":"2024-03-18T15:33:43.994431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:41:28.732114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.995320+0000","last_change":"2024-03-18T15:33:42.998067+0000","last_active":"2024-03-18T15:33:43.995320+0000","last_peered":"2024-03-18T15:33:43.995320+0000","last_clean":"2024-03-18T15:33:43.995320+0000","last_became_active":"2024-03-18T15:33:42.997755+0000","last_became_peered":"2024-03-18T15:33:42.997755+0000","last_unstale":"2024-03-18T15:33:43.995320+0000","last_undegraded":"2024-03-18T15:33:43.995320+0000","last_fullsized":"2024-03-18T15:33:43.995320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:10:01.824296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.869684+0000","last_change":"2024-03-18T15:33:41.987009+0000","last_active":"2024-03-18T15:33:44.869684+0000","last_peered":"2024-03-18T15:33:44.869684+0000","last_clean":"2024-03-18T15:33:44.869684+0000","last_became_active":"2024-03-18T15:33:41.986376+0000","last_became_peered":"2024-03-18T15:33:41.986376+0000","last_unstale":"2024-03-18T15:33:44.869684+0000","last_undegraded":"2024-03-18T15:33:44.869684+0000","last_fullsized":"2024-03-18T15:33:44.869684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:26:02.875362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.994960+0000","last_change":"2024-03-18T15:33:43.007612+0000","last_active":"2024-03-18T15:33:43.994960+0000","last_peered":"2024-03-18T15:33:43.994960+0000","last_clean":"2024-03-18T15:33:43.994960+0000","last_became_active":"2024-03-18T15:33:43.006950+0000","last_became_peered":"2024-03-18T15:33:43.006950+0000","last_unstale":"2024-03-18T15:33:43.994960+0000","last_undegraded":"2024-03-18T15:33:43.994960+0000","last_fullsized":"2024-03-18T15:33:43.994960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:52:46.912729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.996448+0000","last_change":"2024-03-18T15:33:41.990256+0000","last_active":"2024-03-18T15:33:43.996448+0000","last_peered":"2024-03-18T15:33:43.996448+0000","last_clean":"2024-03-18T15:33:43.996448+0000","last_became_active":"2024-03-18T15:33:41.989715+0000","last_became_peered":"2024-03-18T15:33:41.989715+0000","last_unstale":"2024-03-18T15:33:43.996448+0000","last_undegraded":"2024-03-18T15:33:43.996448+0000","last_fullsized":"2024-03-18T15:33:43.996448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:23:20.085212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.096441+0000","last_change":"2024-03-18T15:33:43.022519+0000","last_active":"2024-03-18T15:33:45.096441+0000","last_peered":"2024-03-18T15:33:45.096441+0000","last_clean":"2024-03-18T15:33:45.096441+0000","last_became_active":"2024-03-18T15:33:43.022081+0000","last_became_peered":"2024-03-18T15:33:43.022081+0000","last_unstale":"2024-03-18T15:33:45.096441+0000","last_undegraded":"2024-03-18T15:33:45.096441+0000","last_fullsized":"2024-03-18T15:33:45.096441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:09:14.711960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.869440+0000","last_change":"2024-03-18T15:33:42.001343+0000","last_active":"2024-03-18T15:33:44.869440+0000","last_peered":"2024-03-18T15:33:44.869440+0000","last_clean":"2024-03-18T15:33:44.869440+0000","last_became_active":"2024-03-18T15:33:42.000948+0000","last_became_peered":"2024-03-18T15:33:42.000948+0000","last_unstale":"2024-03-18T15:33:44.869440+0000","last_undegraded":"2024-03-18T15:33:44.869440+0000","last_fullsized":"2024-03-18T15:33:44.869440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:47:53.438385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.993920+0000","last_change":"2024-03-18T15:33:43.015405+0000","last_active":"2024-03-18T15:33:43.993920+0000","last_peered":"2024-03-18T15:33:43.993920+0000","last_clean":"2024-03-18T15:33:43.993920+0000","last_became_active":"2024-03-18T15:33:43.015094+0000","last_became_peered":"2024-03-18T15:33:43.015094+0000","last_unstale":"2024-03-18T15:33:43.993920+0000","last_undegraded":"2024-03-18T15:33:43.993920+0000","last_fullsized":"2024-03-18T15:33:43.993920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:30:16.022611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.993319+0000","last_change":"2024-03-18T15:33:30.766315+0000","last_active":"2024-03-18T15:33:43.993319+0000","last_peered":"2024-03-18T15:33:43.993319+0000","last_clean":"2024-03-18T15:33:43.993319+0000","last_became_active":"2024-03-18T15:33:30.763939+0000","last_became_peered":"2024-03-18T15:33:30.763939+0000","last_unstale":"2024-03-18T15:33:43.993319+0000","last_undegraded":"2024-03-18T15:33:43.993319+0000","last_fullsized":"2024-03-18T15:33:43.993319+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:29.229911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:29.229911+0000","last_clean_scrub_stamp":"2024-03-18T15:33:29.229911+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:10:21.120371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.095640+0000","last_change":"2024-03-18T15:33:41.998201+0000","last_active":"2024-03-18T15:33:45.095640+0000","last_peered":"2024-03-18T15:33:45.095640+0000","last_clean":"2024-03-18T15:33:45.095640+0000","last_became_active":"2024-03-18T15:33:41.997079+0000","last_became_peered":"2024-03-18T15:33:41.997079+0000","last_unstale":"2024-03-18T15:33:45.095640+0000","last_undegraded":"2024-03-18T15:33:45.095640+0000","last_fullsized":"2024-03-18T15:33:45.095640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:38:45.057586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.097810+0000","last_change":"2024-03-18T15:33:43.019262+0000","last_active":"2024-03-18T15:33:45.097810+0000","last_peered":"2024-03-18T15:33:45.097810+0000","last_clean":"2024-03-18T15:33:45.097810+0000","last_became_active":"2024-03-18T15:33:43.018918+0000","last_became_peered":"2024-03-18T15:33:43.018918+0000","last_unstale":"2024-03-18T15:33:45.097810+0000","last_undegraded":"2024-03-18T15:33:45.097810+0000","last_fullsized":"2024-03-18T15:33:45.097810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:16:44.575392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.993340+0000","last_change":"2024-03-18T15:33:42.003947+0000","last_active":"2024-03-18T15:33:43.993340+0000","last_peered":"2024-03-18T15:33:43.993340+0000","last_clean":"2024-03-18T15:33:43.993340+0000","last_became_active":"2024-03-18T15:33:42.002250+0000","last_became_peered":"2024-03-18T15:33:42.002250+0000","last_unstale":"2024-03-18T15:33:43.993340+0000","last_undegraded":"2024-03-18T15:33:43.993340+0000","last_fullsized":"2024-03-18T15:33:43.993340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:07:04.089081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.096070+0000","last_change":"2024-03-18T15:33:43.015094+0000","last_active":"2024-03-18T15:33:45.096070+0000","last_peered":"2024-03-18T15:33:45.096070+0000","last_clean":"2024-03-18T15:33:45.096070+0000","last_became_active":"2024-03-18T15:33:43.014596+0000","last_became_peered":"2024-03-18T15:33:43.014596+0000","last_unstale":"2024-03-18T15:33:45.096070+0000","last_undegraded":"2024-03-18T15:33:45.096070+0000","last_fullsized":"2024-03-18T15:33:45.096070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:31:24.037103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.869444+0000","last_change":"2024-03-18T15:33:41.991149+0000","last_active":"2024-03-18T15:33:44.869444+0000","last_peered":"2024-03-18T15:33:44.869444+0000","last_clean":"2024-03-18T15:33:44.869444+0000","last_became_active":"2024-03-18T15:33:41.990059+0000","last_became_peered":"2024-03-18T15:33:41.990059+0000","last_unstale":"2024-03-18T15:33:44.869444+0000","last_undegraded":"2024-03-18T15:33:44.869444+0000","last_fullsized":"2024-03-18T15:33:44.869444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:09:14.869832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.995704+0000","last_change":"2024-03-18T15:33:43.000746+0000","last_active":"2024-03-18T15:33:43.995704+0000","last_peered":"2024-03-18T15:33:43.995704+0000","last_clean":"2024-03-18T15:33:43.995704+0000","last_became_active":"2024-03-18T15:33:42.999935+0000","last_became_peered":"2024-03-18T15:33:42.999935+0000","last_unstale":"2024-03-18T15:33:43.995704+0000","last_undegraded":"2024-03-18T15:33:43.995704+0000","last_fullsized":"2024-03-18T15:33:43.995704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:11:30.652938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.096162+0000","last_change":"2024-03-18T15:33:42.002409+0000","last_active":"2024-03-18T15:33:45.096162+0000","last_peered":"2024-03-18T15:33:45.096162+0000","last_clean":"2024-03-18T15:33:45.096162+0000","last_became_active":"2024-03-18T15:33:42.000616+0000","last_became_peered":"2024-03-18T15:33:42.000616+0000","last_unstale":"2024-03-18T15:33:45.096162+0000","last_undegraded":"2024-03-18T15:33:45.096162+0000","last_fullsized":"2024-03-18T15:33:45.096162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:47:53.438385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.994828+0000","last_change":"2024-03-18T15:33:43.005420+0000","last_active":"2024-03-18T15:33:43.994828+0000","last_peered":"2024-03-18T15:33:43.994828+0000","last_clean":"2024-03-18T15:33:43.994828+0000","last_became_active":"2024-03-18T15:33:43.004488+0000","last_became_peered":"2024-03-18T15:33:43.004488+0000","last_unstale":"2024-03-18T15:33:43.994828+0000","last_undegraded":"2024-03-18T15:33:43.994828+0000","last_fullsized":"2024-03-18T15:33:43.994828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:35:46.663732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.996031+0000","last_change":"2024-03-18T15:33:41.990472+0000","last_active":"2024-03-18T15:33:43.996031+0000","last_peered":"2024-03-18T15:33:43.996031+0000","last_clean":"2024-03-18T15:33:43.996031+0000","last_became_active":"2024-03-18T15:33:41.989782+0000","last_became_peered":"2024-03-18T15:33:41.989782+0000","last_unstale":"2024-03-18T15:33:43.996031+0000","last_undegraded":"2024-03-18T15:33:43.996031+0000","last_fullsized":"2024-03-18T15:33:43.996031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:35:45.651780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.096233+0000","last_change":"2024-03-18T15:33:43.013011+0000","last_active":"2024-03-18T15:33:45.096233+0000","last_peered":"2024-03-18T15:33:45.096233+0000","last_clean":"2024-03-18T15:33:45.096233+0000","last_became_active":"2024-03-18T15:33:43.012334+0000","last_became_peered":"2024-03-18T15:33:43.012334+0000","last_unstale":"2024-03-18T15:33:45.096233+0000","last_undegraded":"2024-03-18T15:33:45.096233+0000","last_fullsized":"2024-03-18T15:33:45.096233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:43:15.113425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.994085+0000","last_change":"2024-03-18T15:33:42.011960+0000","last_active":"2024-03-18T15:33:43.994085+0000","last_peered":"2024-03-18T15:33:43.994085+0000","last_clean":"2024-03-18T15:33:43.994085+0000","last_became_active":"2024-03-18T15:33:42.010760+0000","last_became_peered":"2024-03-18T15:33:42.010760+0000","last_unstale":"2024-03-18T15:33:43.994085+0000","last_undegraded":"2024-03-18T15:33:43.994085+0000","last_fullsized":"2024-03-18T15:33:43.994085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:29:33.713515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.996001+0000","last_change":"2024-03-18T15:33:42.985474+0000","last_active":"2024-03-18T15:33:43.996001+0000","last_peered":"2024-03-18T15:33:43.996001+0000","last_clean":"2024-03-18T15:33:43.996001+0000","last_became_active":"2024-03-18T15:33:42.984659+0000","last_became_peered":"2024-03-18T15:33:42.984659+0000","last_unstale":"2024-03-18T15:33:43.996001+0000","last_undegraded":"2024-03-18T15:33:43.996001+0000","last_fullsized":"2024-03-18T15:33:43.996001+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:38:08.174152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.996599+0000","last_change":"2024-03-18T15:33:41.994297+0000","last_active":"2024-03-18T15:33:43.996599+0000","last_peered":"2024-03-18T15:33:43.996599+0000","last_clean":"2024-03-18T15:33:43.996599+0000","last_became_active":"2024-03-18T15:33:41.993906+0000","last_became_peered":"2024-03-18T15:33:41.993906+0000","last_unstale":"2024-03-18T15:33:43.996599+0000","last_undegraded":"2024-03-18T15:33:43.996599+0000","last_fullsized":"2024-03-18T15:33:43.996599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:07:04.089081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.099121+0000","last_change":"2024-03-18T15:33:43.018039+0000","last_active":"2024-03-18T15:33:45.099121+0000","last_peered":"2024-03-18T15:33:45.099121+0000","last_clean":"2024-03-18T15:33:45.099121+0000","last_became_active":"2024-03-18T15:33:43.017680+0000","last_became_peered":"2024-03-18T15:33:43.017680+0000","last_unstale":"2024-03-18T15:33:45.099121+0000","last_undegraded":"2024-03-18T15:33:45.099121+0000","last_fullsized":"2024-03-18T15:33:45.099121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:29:15.496652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.002591+0000","last_change":"2024-03-18T15:33:42.009920+0000","last_active":"2024-03-18T15:33:44.002591+0000","last_peered":"2024-03-18T15:33:44.002591+0000","last_clean":"2024-03-18T15:33:44.002591+0000","last_became_active":"2024-03-18T15:33:42.009394+0000","last_became_peered":"2024-03-18T15:33:42.009394+0000","last_unstale":"2024-03-18T15:33:44.002591+0000","last_undegraded":"2024-03-18T15:33:44.002591+0000","last_fullsized":"2024-03-18T15:33:44.002591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:27:05.573887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.097310+0000","last_change":"2024-03-18T15:33:42.999878+0000","last_active":"2024-03-18T15:33:45.097310+0000","last_peered":"2024-03-18T15:33:45.097310+0000","last_clean":"2024-03-18T15:33:45.097310+0000","last_became_active":"2024-03-18T15:33:42.999140+0000","last_became_peered":"2024-03-18T15:33:42.999140+0000","last_unstale":"2024-03-18T15:33:45.097310+0000","last_undegraded":"2024-03-18T15:33:45.097310+0000","last_fullsized":"2024-03-18T15:33:45.097310+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:28:55.955807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.993177+0000","last_change":"2024-03-18T15:33:42.009110+0000","last_active":"2024-03-18T15:33:43.993177+0000","last_peered":"2024-03-18T15:33:43.993177+0000","last_clean":"2024-03-18T15:33:43.993177+0000","last_became_active":"2024-03-18T15:33:42.008110+0000","last_became_peered":"2024-03-18T15:33:42.008110+0000","last_unstale":"2024-03-18T15:33:43.993177+0000","last_undegraded":"2024-03-18T15:33:43.993177+0000","last_fullsized":"2024-03-18T15:33:43.993177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:23:20.085212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.994867+0000","last_change":"2024-03-18T15:33:43.006171+0000","last_active":"2024-03-18T15:33:43.994867+0000","last_peered":"2024-03-18T15:33:43.994867+0000","last_clean":"2024-03-18T15:33:43.994867+0000","last_became_active":"2024-03-18T15:33:43.005851+0000","last_became_peered":"2024-03-18T15:33:43.005851+0000","last_unstale":"2024-03-18T15:33:43.994867+0000","last_undegraded":"2024-03-18T15:33:43.994867+0000","last_fullsized":"2024-03-18T15:33:43.994867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:23:21.097164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.995361+0000","last_change":"2024-03-18T15:33:41.997380+0000","last_active":"2024-03-18T15:33:43.995361+0000","last_peered":"2024-03-18T15:33:43.995361+0000","last_clean":"2024-03-18T15:33:43.995361+0000","last_became_active":"2024-03-18T15:33:41.997101+0000","last_became_peered":"2024-03-18T15:33:41.997101+0000","last_unstale":"2024-03-18T15:33:43.995361+0000","last_undegraded":"2024-03-18T15:33:43.995361+0000","last_fullsized":"2024-03-18T15:33:43.995361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:33:11.480578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.097164+0000","last_change":"2024-03-18T15:33:43.012575+0000","last_active":"2024-03-18T15:33:45.097164+0000","last_peered":"2024-03-18T15:33:45.097164+0000","last_clean":"2024-03-18T15:33:45.097164+0000","last_became_active":"2024-03-18T15:33:42.996972+0000","last_became_peered":"2024-03-18T15:33:42.996972+0000","last_unstale":"2024-03-18T15:33:45.097164+0000","last_undegraded":"2024-03-18T15:33:45.097164+0000","last_fullsized":"2024-03-18T15:33:45.097164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:08:08.722076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.996259+0000","last_change":"2024-03-18T15:33:41.990440+0000","last_active":"2024-03-18T15:33:43.996259+0000","last_peered":"2024-03-18T15:33:43.996259+0000","last_clean":"2024-03-18T15:33:43.996259+0000","last_became_active":"2024-03-18T15:33:41.989808+0000","last_became_peered":"2024-03-18T15:33:41.989808+0000","last_unstale":"2024-03-18T15:33:43.996259+0000","last_undegraded":"2024-03-18T15:33:43.996259+0000","last_fullsized":"2024-03-18T15:33:43.996259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:14:42.231258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.098120+0000","last_change":"2024-03-18T15:33:43.012652+0000","last_active":"2024-03-18T15:33:45.098120+0000","last_peered":"2024-03-18T15:33:45.098120+0000","last_clean":"2024-03-18T15:33:45.098120+0000","last_became_active":"2024-03-18T15:33:43.010391+0000","last_became_peered":"2024-03-18T15:33:43.010391+0000","last_unstale":"2024-03-18T15:33:45.098120+0000","last_undegraded":"2024-03-18T15:33:45.098120+0000","last_fullsized":"2024-03-18T15:33:45.098120+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:38:08.174152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.994505+0000","last_change":"2024-03-18T15:33:42.013549+0000","last_active":"2024-03-18T15:33:43.994505+0000","last_peered":"2024-03-18T15:33:43.994505+0000","last_clean":"2024-03-18T15:33:43.994505+0000","last_became_active":"2024-03-18T15:33:42.012707+0000","last_became_peered":"2024-03-18T15:33:42.012707+0000","last_unstale":"2024-03-18T15:33:43.994505+0000","last_undegraded":"2024-03-18T15:33:43.994505+0000","last_fullsized":"2024-03-18T15:33:43.994505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:14:42.231258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.002049+0000","last_change":"2024-03-18T15:33:43.003144+0000","last_active":"2024-03-18T15:33:44.002049+0000","last_peered":"2024-03-18T15:33:44.002049+0000","last_clean":"2024-03-18T15:33:44.002049+0000","last_became_active":"2024-03-18T15:33:43.002771+0000","last_became_peered":"2024-03-18T15:33:43.002771+0000","last_unstale":"2024-03-18T15:33:44.002049+0000","last_undegraded":"2024-03-18T15:33:44.002049+0000","last_fullsized":"2024-03-18T15:33:44.002049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:50:43.021257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.096229+0000","last_change":"2024-03-18T15:33:43.014403+0000","last_active":"2024-03-18T15:33:45.096229+0000","last_peered":"2024-03-18T15:33:45.096229+0000","last_clean":"2024-03-18T15:33:45.096229+0000","last_became_active":"2024-03-18T15:33:43.012667+0000","last_became_peered":"2024-03-18T15:33:43.012667+0000","last_unstale":"2024-03-18T15:33:45.096229+0000","last_undegraded":"2024-03-18T15:33:45.096229+0000","last_fullsized":"2024-03-18T15:33:45.096229+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:50:44.996218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.993602+0000","last_change":"2024-03-18T15:33:42.009430+0000","last_active":"2024-03-18T15:33:43.993602+0000","last_peered":"2024-03-18T15:33:43.993602+0000","last_clean":"2024-03-18T15:33:43.993602+0000","last_became_active":"2024-03-18T15:33:42.008331+0000","last_became_peered":"2024-03-18T15:33:42.008331+0000","last_unstale":"2024-03-18T15:33:43.993602+0000","last_undegraded":"2024-03-18T15:33:43.993602+0000","last_fullsized":"2024-03-18T15:33:43.993602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:33:11.480578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.096113+0000","last_change":"2024-03-18T15:33:43.017670+0000","last_active":"2024-03-18T15:33:45.096113+0000","last_peered":"2024-03-18T15:33:45.096113+0000","last_clean":"2024-03-18T15:33:45.096113+0000","last_became_active":"2024-03-18T15:33:43.017297+0000","last_became_peered":"2024-03-18T15:33:43.017297+0000","last_unstale":"2024-03-18T15:33:45.096113+0000","last_undegraded":"2024-03-18T15:33:45.096113+0000","last_fullsized":"2024-03-18T15:33:45.096113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:30:50.311074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.993805+0000","last_change":"2024-03-18T15:33:42.009132+0000","last_active":"2024-03-18T15:33:43.993805+0000","last_peered":"2024-03-18T15:33:43.993805+0000","last_clean":"2024-03-18T15:33:43.993805+0000","last_became_active":"2024-03-18T15:33:42.008343+0000","last_became_peered":"2024-03-18T15:33:42.008343+0000","last_unstale":"2024-03-18T15:33:43.993805+0000","last_undegraded":"2024-03-18T15:33:43.993805+0000","last_fullsized":"2024-03-18T15:33:43.993805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T15:35:22.723629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.097441+0000","last_change":"2024-03-18T15:33:43.021277+0000","last_active":"2024-03-18T15:33:45.097441+0000","last_peered":"2024-03-18T15:33:45.097441+0000","last_clean":"2024-03-18T15:33:45.097441+0000","last_became_active":"2024-03-18T15:33:43.020857+0000","last_became_peered":"2024-03-18T15:33:43.020857+0000","last_unstale":"2024-03-18T15:33:45.097441+0000","last_undegraded":"2024-03-18T15:33:45.097441+0000","last_fullsized":"2024-03-18T15:33:45.097441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:24:37.779979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.098590+0000","last_change":"2024-03-18T15:33:41.992048+0000","last_active":"2024-03-18T15:33:45.098590+0000","last_peered":"2024-03-18T15:33:45.098590+0000","last_clean":"2024-03-18T15:33:45.098590+0000","last_became_active":"2024-03-18T15:33:41.989055+0000","last_became_peered":"2024-03-18T15:33:41.989055+0000","last_unstale":"2024-03-18T15:33:45.098590+0000","last_undegraded":"2024-03-18T15:33:45.098590+0000","last_fullsized":"2024-03-18T15:33:45.098590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:46:33.341037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.098751+0000","last_change":"2024-03-18T15:33:42.996569+0000","last_active":"2024-03-18T15:33:45.098751+0000","last_peered":"2024-03-18T15:33:45.098751+0000","last_clean":"2024-03-18T15:33:45.098751+0000","last_became_active":"2024-03-18T15:33:42.996234+0000","last_became_peered":"2024-03-18T15:33:42.996234+0000","last_unstale":"2024-03-18T15:33:45.098751+0000","last_undegraded":"2024-03-18T15:33:45.098751+0000","last_fullsized":"2024-03-18T15:33:45.098751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:29:18.287234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.097738+0000","last_change":"2024-03-18T15:33:42.010405+0000","last_active":"2024-03-18T15:33:45.097738+0000","last_peered":"2024-03-18T15:33:45.097738+0000","last_clean":"2024-03-18T15:33:45.097738+0000","last_became_active":"2024-03-18T15:33:42.009883+0000","last_became_peered":"2024-03-18T15:33:42.009883+0000","last_unstale":"2024-03-18T15:33:45.097738+0000","last_undegraded":"2024-03-18T15:33:45.097738+0000","last_fullsized":"2024-03-18T15:33:45.097738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:10:06.802810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":75067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":73545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":806912,"data_stored":664405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542152,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":73581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35556,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336284,"statfs":{"total":96636764160,"available":96600354816,"internally_reserved":0,"allocated":221184,"data_stored":75067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35544,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336296,"statfs":{"total":96636764160,"available":96600367104,"internally_reserved":0,"allocated":208896,"data_stored":73455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":73615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36136,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335704,"statfs":{"total":96636764160,"available":96599760896,"internally_reserved":0,"allocated":815104,"data_stored":664461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-18T15:33:49.642 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-18T15:33:49.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.785+0000 7ff9d60f6640 1 Processor -- start 2024-03-18T15:33:49.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.785+0000 7ff9d60f6640 1 -- start start 2024-03-18T15:33:49.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.789+0000 7ff9d60f6640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff9d00b5710 0x7ff9d014a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:49.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.789+0000 7ff9d60f6640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff9d014ae60 0x7ff9d0140ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:49.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.789+0000 7ff9d60f6640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff9d0141430 0x7ff9d00b7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:49.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.789+0000 7ff9d60f6640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7ff9d00b4320 con 0x7ff9d00b5710 2024-03-18T15:33:49.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.789+0000 7ff9d60f6640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7ff9d00b4570 con 0x7ff9d014ae60 2024-03-18T15:33:49.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.789+0000 7ff9d60f6640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7ff9d00ba0c0 con 0x7ff9d0141430 2024-03-18T15:33:49.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.789+0000 7ff9cf7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff9d00b5710 0x7ff9d014a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:49.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.789+0000 7ff9cf7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff9d00b5710 0x7ff9d014a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38634/0 (socket says 172.21.15.5:38634) 2024-03-18T15:33:49.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.789+0000 7ff9cf7fe640 1 -- 172.21.15.5:0/510650438 learned_addr learned my addr 172.21.15.5:0/510650438 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:49.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.789+0000 7ff9cffff640 1 --2- 172.21.15.5:0/510650438 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff9d0141430 0x7ff9d00b7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:49.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.789+0000 7ff9ceffd640 1 --2- 172.21.15.5:0/510650438 >> v2:172.21.15.80:3300/0 conn(0x7ff9d014ae60 0x7ff9d0140ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:49.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.789+0000 7ff9cf7fe640 1 -- 172.21.15.5:0/510650438 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff9d0141430 msgr2=0x7ff9d00b7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:49.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.789+0000 7ff9cf7fe640 1 --2- 172.21.15.5:0/510650438 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff9d0141430 0x7ff9d00b7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.789+0000 7ff9cf7fe640 1 -- 172.21.15.5:0/510650438 >> v2:172.21.15.80:3300/0 conn(0x7ff9d014ae60 msgr2=0x7ff9d0140ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:49.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.789+0000 7ff9cf7fe640 1 --2- 172.21.15.5:0/510650438 >> v2:172.21.15.80:3300/0 conn(0x7ff9d014ae60 0x7ff9d0140ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.789+0000 7ff9cf7fe640 1 -- 172.21.15.5:0/510650438 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9d00b9d50 con 0x7ff9d00b5710 2024-03-18T15:33:49.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.793+0000 7ff9cf7fe640 1 --2- 172.21.15.5:0/510650438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9d00b5710 0x7ff9d014a920 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff9d00b88c0 tx=0x7ff9c001c140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e02e97e2a5348710 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:49.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.793+0000 7ff9ce7fc640 1 -- 172.21.15.5:0/510650438 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c00b0070 con 0x7ff9d00b5710 2024-03-18T15:33:49.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.793+0000 7ff9ce7fc640 1 -- 172.21.15.5:0/510650438 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9c00ab600 con 0x7ff9d00b5710 2024-03-18T15:33:49.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.793+0000 7ff9ce7fc640 1 -- 172.21.15.5:0/510650438 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c00ab9d0 con 0x7ff9d00b5710 2024-03-18T15:33:49.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.793+0000 7ff9d60f6640 1 -- 172.21.15.5:0/510650438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9d00b5710 msgr2=0x7ff9d014a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:49.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.793+0000 7ff9d60f6640 1 --2- 172.21.15.5:0/510650438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9d00b5710 0x7ff9d014a920 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff9d00b88c0 tx=0x7ff9c001c140 comp rx=0 tx=0).stop 2024-03-18T15:33:49.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.793+0000 7ff9d60f6640 1 -- 172.21.15.5:0/510650438 shutdown_connections 2024-03-18T15:33:49.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.793+0000 7ff9d60f6640 1 --2- 172.21.15.5:0/510650438 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff9d0141430 0x7ff9d00b7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.793+0000 7ff9d60f6640 1 --2- 172.21.15.5:0/510650438 >> v2:172.21.15.80:3300/0 conn(0x7ff9d014ae60 0x7ff9d0140ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.793+0000 7ff9d60f6640 1 --2- 172.21.15.5:0/510650438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9d00b5710 0x7ff9d014a920 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.793+0000 7ff9d60f6640 1 -- 172.21.15.5:0/510650438 >> 172.21.15.5:0/510650438 conn(0x7ff9d00673c0 msgr2=0x7ff9d00677d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:49.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.793+0000 7ff9d60f6640 1 -- 172.21.15.5:0/510650438 shutdown_connections 2024-03-18T15:33:49.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.793+0000 7ff9d60f6640 1 -- 172.21.15.5:0/510650438 wait complete. 2024-03-18T15:33:49.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.793+0000 7ff9d60f6640 1 Processor -- start 2024-03-18T15:33:49.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.793+0000 7ff9d60f6640 1 -- start start 2024-03-18T15:33:49.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.797+0000 7ff9d60f6640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9d00b5710 0x7ff9d013b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:49.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.797+0000 7ff9d60f6640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff9d0141430 0x7ff9d01e3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:49.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.797+0000 7ff9d60f6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9d014ae60 0x7ff9d0147630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:49.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.797+0000 7ff9d60f6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d00b50e0 con 0x7ff9d014ae60 2024-03-18T15:33:49.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.797+0000 7ff9d60f6640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d00b5af0 con 0x7ff9d00b5710 2024-03-18T15:33:49.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.797+0000 7ff9d60f6640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7ff9d00b5260 con 0x7ff9d0141430 2024-03-18T15:33:49.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.797+0000 7ff9cffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9d014ae60 0x7ff9d0147630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:49.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.797+0000 7ff9cffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9d014ae60 0x7ff9d0147630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38644/0 (socket says 172.21.15.5:38644) 2024-03-18T15:33:49.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.797+0000 7ff9cffff640 1 -- 172.21.15.5:0/4248623570 learned_addr learned my addr 172.21.15.5:0/4248623570 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:49.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.797+0000 7ff9cf7fe640 1 --2- 172.21.15.5:0/4248623570 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9d00b5710 0x7ff9d013b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:49.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.797+0000 7ff9ceffd640 1 --2- 172.21.15.5:0/4248623570 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff9d0141430 0x7ff9d01e3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:49.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.797+0000 7ff9cffff640 1 -- 172.21.15.5:0/4248623570 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff9d0141430 msgr2=0x7ff9d01e3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:49.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.797+0000 7ff9cffff640 1 --2- 172.21.15.5:0/4248623570 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff9d0141430 0x7ff9d01e3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.797+0000 7ff9cffff640 1 -- 172.21.15.5:0/4248623570 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9d00b5710 msgr2=0x7ff9d013b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:49.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.797+0000 7ff9cffff640 1 --2- 172.21.15.5:0/4248623570 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9d00b5710 0x7ff9d013b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.797+0000 7ff9cffff640 1 -- 172.21.15.5:0/4248623570 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9d013d5f0 con 0x7ff9d014ae60 2024-03-18T15:33:49.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.797+0000 7ff9cffff640 1 --2- 172.21.15.5:0/4248623570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9d014ae60 0x7ff9d0147630 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff9c800efc0 tx=0x7ff9c800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:49.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.797+0000 7ff9c7fff640 1 -- 172.21.15.5:0/4248623570 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8018020 con 0x7ff9d014ae60 2024-03-18T15:33:49.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.797+0000 7ff9c7fff640 1 -- 172.21.15.5:0/4248623570 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9c8014480 con 0x7ff9d014ae60 2024-03-18T15:33:49.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.797+0000 7ff9d60f6640 1 -- 172.21.15.5:0/4248623570 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9d013fcf0 con 0x7ff9d014ae60 2024-03-18T15:33:49.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.797+0000 7ff9c7fff640 1 -- 172.21.15.5:0/4248623570 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8014870 con 0x7ff9d014ae60 2024-03-18T15:33:49.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.797+0000 7ff9d60f6640 1 -- 172.21.15.5:0/4248623570 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9d013c020 con 0x7ff9d014ae60 2024-03-18T15:33:49.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.797+0000 7ff9d60f6640 1 -- 172.21.15.5:0/4248623570 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9d004f550 con 0x7ff9d014ae60 2024-03-18T15:33:49.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.797+0000 7ff9c7fff640 1 -- 172.21.15.5:0/4248623570 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7ff9c801b020 con 0x7ff9d014ae60 2024-03-18T15:33:49.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.797+0000 7ff9c7fff640 1 --2- 172.21.15.5:0/4248623570 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7ff9b0067660 0x7ff9b0069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:49.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.797+0000 7ff9c7fff640 1 -- 172.21.15.5:0/4248623570 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff9c8086210 con 0x7ff9d014ae60 2024-03-18T15:33:49.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.797+0000 7ff9cf7fe640 1 --2- 172.21.15.5:0/4248623570 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7ff9b0067660 0x7ff9b0069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:49.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.797+0000 7ff9cf7fe640 1 --2- 172.21.15.5:0/4248623570 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7ff9b0067660 0x7ff9b0069b20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff9c00ad000 tx=0x7ff9c00a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:49.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.801+0000 7ff9c7fff640 1 -- 172.21.15.5:0/4248623570 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7ff9c8022070 con 0x7ff9d014ae60 2024-03-18T15:33:49.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.953+0000 7ff9d60f6640 1 -- 172.21.15.5:0/4248623570 --> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff9d01406e0 con 0x7ff9b0067660 2024-03-18T15:33:49.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.969+0000 7ff9c7fff640 1 -- 172.21.15.5:0/4248623570 <== mgr.4110 v2:172.21.15.5:6832/3362034860 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251932 (secure 0 0 0) 0x7ff9c00ab500 con 0x7ff9b0067660 2024-03-18T15:33:49.973 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:33:49.974 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-03-18T15:33:49.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.973+0000 7ff9d60f6640 1 -- 172.21.15.5:0/4248623570 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7ff9b0067660 msgr2=0x7ff9b0069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:49.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.973+0000 7ff9d60f6640 1 --2- 172.21.15.5:0/4248623570 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7ff9b0067660 0x7ff9b0069b20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff9c00ad000 tx=0x7ff9c00a6000 comp rx=0 tx=0).stop 2024-03-18T15:33:49.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.973+0000 7ff9d60f6640 1 -- 172.21.15.5:0/4248623570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9d014ae60 msgr2=0x7ff9d0147630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:49.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.973+0000 7ff9d60f6640 1 --2- 172.21.15.5:0/4248623570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9d014ae60 0x7ff9d0147630 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff9c800efc0 tx=0x7ff9c800c490 comp rx=0 tx=0).stop 2024-03-18T15:33:49.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.973+0000 7ff9d60f6640 1 -- 172.21.15.5:0/4248623570 shutdown_connections 2024-03-18T15:33:49.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.973+0000 7ff9d60f6640 1 --2- 172.21.15.5:0/4248623570 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7ff9b0067660 0x7ff9b0069b20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.973+0000 7ff9d60f6640 1 --2- 172.21.15.5:0/4248623570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9d014ae60 0x7ff9d0147630 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.973+0000 7ff9d60f6640 1 --2- 172.21.15.5:0/4248623570 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff9d0141430 0x7ff9d01e3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.973+0000 7ff9d60f6640 1 --2- 172.21.15.5:0/4248623570 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9d00b5710 0x7ff9d013b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:49.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.973+0000 7ff9d60f6640 1 -- 172.21.15.5:0/4248623570 >> 172.21.15.5:0/4248623570 conn(0x7ff9d00673c0 msgr2=0x7ff9d0136f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:49.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.973+0000 7ff9d60f6640 1 -- 172.21.15.5:0/4248623570 shutdown_connections 2024-03-18T15:33:49.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:49.973+0000 7ff9d60f6640 1 -- 172.21.15.5:0/4248623570 wait complete. 2024-03-18T15:33:49.990 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-03-18T15:33:48.887212+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":248160,"kb_used_data":2856,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754726560,"statfs":{"total":773094113280,"available":772839997440,"internally_reserved":0,"allocated":2924544,"data_stored":1773196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.922001"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.994364+0000","last_change":"2024-03-18T15:33:42.998484+0000","last_active":"2024-03-18T15:33:43.994364+0000","last_peered":"2024-03-18T15:33:43.994364+0000","last_clean":"2024-03-18T15:33:43.994364+0000","last_became_active":"2024-03-18T15:33:42.996054+0000","last_became_peered":"2024-03-18T15:33:42.996054+0000","last_unstale":"2024-03-18T15:33:43.994364+0000","last_undegraded":"2024-03-18T15:33:43.994364+0000","last_fullsized":"2024-03-18T15:33:43.994364+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:38:03.814592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.996595+0000","last_change":"2024-03-18T15:33:42.997522+0000","last_active":"2024-03-18T15:33:43.996595+0000","last_peered":"2024-03-18T15:33:43.996595+0000","last_clean":"2024-03-18T15:33:43.996595+0000","last_became_active":"2024-03-18T15:33:42.997164+0000","last_became_peered":"2024-03-18T15:33:42.997164+0000","last_unstale":"2024-03-18T15:33:43.996595+0000","last_undegraded":"2024-03-18T15:33:43.996595+0000","last_fullsized":"2024-03-18T15:33:43.996595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:29:18.287234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.096933+0000","last_change":"2024-03-18T15:33:43.001309+0000","last_active":"2024-03-18T15:33:45.096933+0000","last_peered":"2024-03-18T15:33:45.096933+0000","last_clean":"2024-03-18T15:33:45.096933+0000","last_became_active":"2024-03-18T15:33:43.000919+0000","last_became_peered":"2024-03-18T15:33:43.000919+0000","last_unstale":"2024-03-18T15:33:45.096933+0000","last_undegraded":"2024-03-18T15:33:45.096933+0000","last_fullsized":"2024-03-18T15:33:45.096933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:10:31.661870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.095996+0000","last_change":"2024-03-18T15:33:43.022575+0000","last_active":"2024-03-18T15:33:45.095996+0000","last_peered":"2024-03-18T15:33:45.095996+0000","last_clean":"2024-03-18T15:33:45.095996+0000","last_became_active":"2024-03-18T15:33:43.022221+0000","last_became_peered":"2024-03-18T15:33:43.022221+0000","last_unstale":"2024-03-18T15:33:45.095996+0000","last_undegraded":"2024-03-18T15:33:45.095996+0000","last_fullsized":"2024-03-18T15:33:45.095996+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:59:03.138076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.994821+0000","last_change":"2024-03-18T15:33:42.999245+0000","last_active":"2024-03-18T15:33:43.994821+0000","last_peered":"2024-03-18T15:33:43.994821+0000","last_clean":"2024-03-18T15:33:43.994821+0000","last_became_active":"2024-03-18T15:33:42.998951+0000","last_became_peered":"2024-03-18T15:33:42.998951+0000","last_unstale":"2024-03-18T15:33:43.994821+0000","last_undegraded":"2024-03-18T15:33:43.994821+0000","last_fullsized":"2024-03-18T15:33:43.994821+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:51:36.500416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.095710+0000","last_change":"2024-03-18T15:33:43.017400+0000","last_active":"2024-03-18T15:33:45.095710+0000","last_peered":"2024-03-18T15:33:45.095710+0000","last_clean":"2024-03-18T15:33:45.095710+0000","last_became_active":"2024-03-18T15:33:43.017074+0000","last_became_peered":"2024-03-18T15:33:43.017074+0000","last_unstale":"2024-03-18T15:33:45.095710+0000","last_undegraded":"2024-03-18T15:33:45.095710+0000","last_fullsized":"2024-03-18T15:33:45.095710+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:50:28.968807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.993424+0000","last_change":"2024-03-18T15:33:42.998214+0000","last_active":"2024-03-18T15:33:43.993424+0000","last_peered":"2024-03-18T15:33:43.993424+0000","last_clean":"2024-03-18T15:33:43.993424+0000","last_became_active":"2024-03-18T15:33:42.997922+0000","last_became_peered":"2024-03-18T15:33:42.997922+0000","last_unstale":"2024-03-18T15:33:43.993424+0000","last_undegraded":"2024-03-18T15:33:43.993424+0000","last_fullsized":"2024-03-18T15:33:43.993424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:53:24.132151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.096323+0000","last_change":"2024-03-18T15:33:43.021183+0000","last_active":"2024-03-18T15:33:45.096323+0000","last_peered":"2024-03-18T15:33:45.096323+0000","last_clean":"2024-03-18T15:33:45.096323+0000","last_became_active":"2024-03-18T15:33:43.020848+0000","last_became_peered":"2024-03-18T15:33:43.020848+0000","last_unstale":"2024-03-18T15:33:45.096323+0000","last_undegraded":"2024-03-18T15:33:45.096323+0000","last_fullsized":"2024-03-18T15:33:45.096323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:06:40.477304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.994576+0000","last_change":"2024-03-18T15:33:43.018236+0000","last_active":"2024-03-18T15:33:43.994576+0000","last_peered":"2024-03-18T15:33:43.994576+0000","last_clean":"2024-03-18T15:33:43.994576+0000","last_became_active":"2024-03-18T15:33:43.017781+0000","last_became_peered":"2024-03-18T15:33:43.017781+0000","last_unstale":"2024-03-18T15:33:43.994576+0000","last_undegraded":"2024-03-18T15:33:43.994576+0000","last_fullsized":"2024-03-18T15:33:43.994576+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:57:09.423392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.002529+0000","last_change":"2024-03-18T15:33:43.020001+0000","last_active":"2024-03-18T15:33:44.002529+0000","last_peered":"2024-03-18T15:33:44.002529+0000","last_clean":"2024-03-18T15:33:44.002529+0000","last_became_active":"2024-03-18T15:33:43.019640+0000","last_became_peered":"2024-03-18T15:33:43.019640+0000","last_unstale":"2024-03-18T15:33:44.002529+0000","last_undegraded":"2024-03-18T15:33:44.002529+0000","last_fullsized":"2024-03-18T15:33:44.002529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:28:55.955807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.002450+0000","last_change":"2024-03-18T15:33:43.020077+0000","last_active":"2024-03-18T15:33:44.002450+0000","last_peered":"2024-03-18T15:33:44.002450+0000","last_clean":"2024-03-18T15:33:44.002450+0000","last_became_active":"2024-03-18T15:33:43.019703+0000","last_became_peered":"2024-03-18T15:33:43.019703+0000","last_unstale":"2024-03-18T15:33:44.002450+0000","last_undegraded":"2024-03-18T15:33:44.002450+0000","last_fullsized":"2024-03-18T15:33:44.002450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:10:31.661870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.994862+0000","last_change":"2024-03-18T15:33:42.988744+0000","last_active":"2024-03-18T15:33:43.994862+0000","last_peered":"2024-03-18T15:33:43.994862+0000","last_clean":"2024-03-18T15:33:43.994862+0000","last_became_active":"2024-03-18T15:33:42.988355+0000","last_became_peered":"2024-03-18T15:33:42.988355+0000","last_unstale":"2024-03-18T15:33:43.994862+0000","last_undegraded":"2024-03-18T15:33:43.994862+0000","last_fullsized":"2024-03-18T15:33:43.994862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:43:42.420536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.098025+0000","last_change":"2024-03-18T15:33:43.010027+0000","last_active":"2024-03-18T15:33:45.098025+0000","last_peered":"2024-03-18T15:33:45.098025+0000","last_clean":"2024-03-18T15:33:45.098025+0000","last_became_active":"2024-03-18T15:33:43.009401+0000","last_became_peered":"2024-03-18T15:33:43.009401+0000","last_unstale":"2024-03-18T15:33:45.098025+0000","last_undegraded":"2024-03-18T15:33:45.098025+0000","last_fullsized":"2024-03-18T15:33:45.098025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:11:30.652938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.096064+0000","last_change":"2024-03-18T15:33:42.999478+0000","last_active":"2024-03-18T15:33:45.096064+0000","last_peered":"2024-03-18T15:33:45.096064+0000","last_clean":"2024-03-18T15:33:45.096064+0000","last_became_active":"2024-03-18T15:33:42.998918+0000","last_became_peered":"2024-03-18T15:33:42.998918+0000","last_unstale":"2024-03-18T15:33:45.096064+0000","last_undegraded":"2024-03-18T15:33:45.096064+0000","last_fullsized":"2024-03-18T15:33:45.096064+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:48:33.731710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.995511+0000","last_change":"2024-03-18T15:33:42.983262+0000","last_active":"2024-03-18T15:33:43.995511+0000","last_peered":"2024-03-18T15:33:43.995511+0000","last_clean":"2024-03-18T15:33:43.995511+0000","last_became_active":"2024-03-18T15:33:42.982356+0000","last_became_peered":"2024-03-18T15:33:42.982356+0000","last_unstale":"2024-03-18T15:33:43.995511+0000","last_undegraded":"2024-03-18T15:33:43.995511+0000","last_fullsized":"2024-03-18T15:33:43.995511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:16:44.575392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.095621+0000","last_change":"2024-03-18T15:33:43.017589+0000","last_active":"2024-03-18T15:33:45.095621+0000","last_peered":"2024-03-18T15:33:45.095621+0000","last_clean":"2024-03-18T15:33:45.095621+0000","last_became_active":"2024-03-18T15:33:43.016991+0000","last_became_peered":"2024-03-18T15:33:43.016991+0000","last_unstale":"2024-03-18T15:33:45.095621+0000","last_undegraded":"2024-03-18T15:33:45.095621+0000","last_fullsized":"2024-03-18T15:33:45.095621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:37:33.748962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.096964+0000","last_change":"2024-03-18T15:33:43.015021+0000","last_active":"2024-03-18T15:33:45.096964+0000","last_peered":"2024-03-18T15:33:45.096964+0000","last_clean":"2024-03-18T15:33:45.096964+0000","last_became_active":"2024-03-18T15:33:43.012689+0000","last_became_peered":"2024-03-18T15:33:43.012689+0000","last_unstale":"2024-03-18T15:33:45.096964+0000","last_undegraded":"2024-03-18T15:33:45.096964+0000","last_fullsized":"2024-03-18T15:33:45.096964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:19:40.976122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.869526+0000","last_change":"2024-03-18T15:33:42.997254+0000","last_active":"2024-03-18T15:33:44.869526+0000","last_peered":"2024-03-18T15:33:44.869526+0000","last_clean":"2024-03-18T15:33:44.869526+0000","last_became_active":"2024-03-18T15:33:42.995704+0000","last_became_peered":"2024-03-18T15:33:42.995704+0000","last_unstale":"2024-03-18T15:33:44.869526+0000","last_undegraded":"2024-03-18T15:33:44.869526+0000","last_fullsized":"2024-03-18T15:33:44.869526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:13:27.226842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.994025+0000","last_change":"2024-03-18T15:33:42.998872+0000","last_active":"2024-03-18T15:33:43.994025+0000","last_peered":"2024-03-18T15:33:43.994025+0000","last_clean":"2024-03-18T15:33:43.994025+0000","last_became_active":"2024-03-18T15:33:42.998529+0000","last_became_peered":"2024-03-18T15:33:42.998529+0000","last_unstale":"2024-03-18T15:33:43.994025+0000","last_undegraded":"2024-03-18T15:33:43.994025+0000","last_fullsized":"2024-03-18T15:33:43.994025+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:00:41.383860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.002213+0000","last_change":"2024-03-18T15:33:43.003843+0000","last_active":"2024-03-18T15:33:44.002213+0000","last_peered":"2024-03-18T15:33:44.002213+0000","last_clean":"2024-03-18T15:33:44.002213+0000","last_became_active":"2024-03-18T15:33:43.003481+0000","last_became_peered":"2024-03-18T15:33:43.003481+0000","last_unstale":"2024-03-18T15:33:44.002213+0000","last_undegraded":"2024-03-18T15:33:44.002213+0000","last_fullsized":"2024-03-18T15:33:44.002213+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:21:20.566649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.096775+0000","last_change":"2024-03-18T15:33:43.016217+0000","last_active":"2024-03-18T15:33:45.096775+0000","last_peered":"2024-03-18T15:33:45.096775+0000","last_clean":"2024-03-18T15:33:45.096775+0000","last_became_active":"2024-03-18T15:33:43.015511+0000","last_became_peered":"2024-03-18T15:33:43.015511+0000","last_unstale":"2024-03-18T15:33:45.096775+0000","last_undegraded":"2024-03-18T15:33:45.096775+0000","last_fullsized":"2024-03-18T15:33:45.096775+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:50:43.021257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.095772+0000","last_change":"2024-03-18T15:33:43.014558+0000","last_active":"2024-03-18T15:33:45.095772+0000","last_peered":"2024-03-18T15:33:45.095772+0000","last_clean":"2024-03-18T15:33:45.095772+0000","last_became_active":"2024-03-18T15:33:43.012745+0000","last_became_peered":"2024-03-18T15:33:43.012745+0000","last_unstale":"2024-03-18T15:33:45.095772+0000","last_undegraded":"2024-03-18T15:33:45.095772+0000","last_fullsized":"2024-03-18T15:33:45.095772+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:24:37.779979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.097432+0000","last_change":"2024-03-18T15:33:43.010632+0000","last_active":"2024-03-18T15:33:45.097432+0000","last_peered":"2024-03-18T15:33:45.097432+0000","last_clean":"2024-03-18T15:33:45.097432+0000","last_became_active":"2024-03-18T15:33:43.010198+0000","last_became_peered":"2024-03-18T15:33:43.010198+0000","last_unstale":"2024-03-18T15:33:45.097432+0000","last_undegraded":"2024-03-18T15:33:45.097432+0000","last_fullsized":"2024-03-18T15:33:45.097432+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:03:19.737667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.095654+0000","last_change":"2024-03-18T15:33:43.018016+0000","last_active":"2024-03-18T15:33:45.095654+0000","last_peered":"2024-03-18T15:33:45.095654+0000","last_clean":"2024-03-18T15:33:45.095654+0000","last_became_active":"2024-03-18T15:33:43.017655+0000","last_became_peered":"2024-03-18T15:33:43.017655+0000","last_unstale":"2024-03-18T15:33:45.095654+0000","last_undegraded":"2024-03-18T15:33:45.095654+0000","last_fullsized":"2024-03-18T15:33:45.095654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T15:41:33.909895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.099107+0000","last_change":"2024-03-18T15:33:43.010776+0000","last_active":"2024-03-18T15:33:45.099107+0000","last_peered":"2024-03-18T15:33:45.099107+0000","last_clean":"2024-03-18T15:33:45.099107+0000","last_became_active":"2024-03-18T15:33:43.010222+0000","last_became_peered":"2024-03-18T15:33:43.010222+0000","last_unstale":"2024-03-18T15:33:45.099107+0000","last_undegraded":"2024-03-18T15:33:45.099107+0000","last_fullsized":"2024-03-18T15:33:45.099107+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:53:24.132151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.002604+0000","last_change":"2024-03-18T15:33:43.011621+0000","last_active":"2024-03-18T15:33:44.002604+0000","last_peered":"2024-03-18T15:33:44.002604+0000","last_clean":"2024-03-18T15:33:44.002604+0000","last_became_active":"2024-03-18T15:33:43.011354+0000","last_became_peered":"2024-03-18T15:33:43.011354+0000","last_unstale":"2024-03-18T15:33:44.002604+0000","last_undegraded":"2024-03-18T15:33:44.002604+0000","last_fullsized":"2024-03-18T15:33:44.002604+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:50:28.968807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.996192+0000","last_change":"2024-03-18T15:33:43.011713+0000","last_active":"2024-03-18T15:33:43.996192+0000","last_peered":"2024-03-18T15:33:43.996192+0000","last_clean":"2024-03-18T15:33:43.996192+0000","last_became_active":"2024-03-18T15:33:43.011288+0000","last_became_peered":"2024-03-18T15:33:43.011288+0000","last_unstale":"2024-03-18T15:33:43.996192+0000","last_undegraded":"2024-03-18T15:33:43.996192+0000","last_fullsized":"2024-03-18T15:33:43.996192+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:07:37.987066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.098603+0000","last_change":"2024-03-18T15:33:43.018468+0000","last_active":"2024-03-18T15:33:45.098603+0000","last_peered":"2024-03-18T15:33:45.098603+0000","last_clean":"2024-03-18T15:33:45.098603+0000","last_became_active":"2024-03-18T15:33:43.012834+0000","last_became_peered":"2024-03-18T15:33:43.012834+0000","last_unstale":"2024-03-18T15:33:45.098603+0000","last_undegraded":"2024-03-18T15:33:45.098603+0000","last_fullsized":"2024-03-18T15:33:45.098603+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:07:37.987066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.096707+0000","last_change":"2024-03-18T15:33:43.016590+0000","last_active":"2024-03-18T15:33:45.096707+0000","last_peered":"2024-03-18T15:33:45.096707+0000","last_clean":"2024-03-18T15:33:45.096707+0000","last_became_active":"2024-03-18T15:33:43.015944+0000","last_became_peered":"2024-03-18T15:33:43.015944+0000","last_unstale":"2024-03-18T15:33:45.096707+0000","last_undegraded":"2024-03-18T15:33:45.096707+0000","last_fullsized":"2024-03-18T15:33:45.096707+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:57:58.192115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.096097+0000","last_change":"2024-03-18T15:33:43.014278+0000","last_active":"2024-03-18T15:33:45.096097+0000","last_peered":"2024-03-18T15:33:45.096097+0000","last_clean":"2024-03-18T15:33:45.096097+0000","last_became_active":"2024-03-18T15:33:43.012307+0000","last_became_peered":"2024-03-18T15:33:43.012307+0000","last_unstale":"2024-03-18T15:33:45.096097+0000","last_undegraded":"2024-03-18T15:33:45.096097+0000","last_fullsized":"2024-03-18T15:33:45.096097+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:54:41.008899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.869746+0000","last_change":"2024-03-18T15:33:43.018853+0000","last_active":"2024-03-18T15:33:44.869746+0000","last_peered":"2024-03-18T15:33:44.869746+0000","last_clean":"2024-03-18T15:33:44.869746+0000","last_became_active":"2024-03-18T15:33:43.018496+0000","last_became_peered":"2024-03-18T15:33:43.018496+0000","last_unstale":"2024-03-18T15:33:44.869746+0000","last_undegraded":"2024-03-18T15:33:44.869746+0000","last_fullsized":"2024-03-18T15:33:44.869746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:43:00.095618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.002017+0000","last_change":"2024-03-18T15:33:43.000804+0000","last_active":"2024-03-18T15:33:44.002017+0000","last_peered":"2024-03-18T15:33:44.002017+0000","last_clean":"2024-03-18T15:33:44.002017+0000","last_became_active":"2024-03-18T15:33:43.000463+0000","last_became_peered":"2024-03-18T15:33:43.000463+0000","last_unstale":"2024-03-18T15:33:44.002017+0000","last_undegraded":"2024-03-18T15:33:44.002017+0000","last_fullsized":"2024-03-18T15:33:44.002017+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:57:58.192115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.995711+0000","last_change":"2024-03-18T15:33:42.995621+0000","last_active":"2024-03-18T15:33:43.995711+0000","last_peered":"2024-03-18T15:33:43.995711+0000","last_clean":"2024-03-18T15:33:43.995711+0000","last_became_active":"2024-03-18T15:33:42.995289+0000","last_became_peered":"2024-03-18T15:33:42.995289+0000","last_unstale":"2024-03-18T15:33:43.995711+0000","last_undegraded":"2024-03-18T15:33:43.995711+0000","last_fullsized":"2024-03-18T15:33:43.995711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:07:05.101032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.098186+0000","last_change":"2024-03-18T15:33:42.010459+0000","last_active":"2024-03-18T15:33:45.098186+0000","last_peered":"2024-03-18T15:33:45.098186+0000","last_clean":"2024-03-18T15:33:45.098186+0000","last_became_active":"2024-03-18T15:33:42.009978+0000","last_became_peered":"2024-03-18T15:33:42.009978+0000","last_unstale":"2024-03-18T15:33:45.098186+0000","last_undegraded":"2024-03-18T15:33:45.098186+0000","last_fullsized":"2024-03-18T15:33:45.098186+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:20:19.179099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.096910+0000","last_change":"2024-03-18T15:33:43.015925+0000","last_active":"2024-03-18T15:33:45.096910+0000","last_peered":"2024-03-18T15:33:45.096910+0000","last_clean":"2024-03-18T15:33:45.096910+0000","last_became_active":"2024-03-18T15:33:43.014716+0000","last_became_peered":"2024-03-18T15:33:43.014716+0000","last_unstale":"2024-03-18T15:33:45.096910+0000","last_undegraded":"2024-03-18T15:33:45.096910+0000","last_fullsized":"2024-03-18T15:33:45.096910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:21:20.566649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.002101+0000","last_change":"2024-03-18T15:33:42.012969+0000","last_active":"2024-03-18T15:33:44.002101+0000","last_peered":"2024-03-18T15:33:44.002101+0000","last_clean":"2024-03-18T15:33:44.002101+0000","last_became_active":"2024-03-18T15:33:42.012416+0000","last_became_peered":"2024-03-18T15:33:42.012416+0000","last_unstale":"2024-03-18T15:33:44.002101+0000","last_undegraded":"2024-03-18T15:33:44.002101+0000","last_fullsized":"2024-03-18T15:33:44.002101+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:47:53.438385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.097620+0000","last_change":"2024-03-18T15:33:41.996706+0000","last_active":"2024-03-18T15:33:45.097620+0000","last_peered":"2024-03-18T15:33:45.097620+0000","last_clean":"2024-03-18T15:33:45.097620+0000","last_became_active":"2024-03-18T15:33:41.996414+0000","last_became_peered":"2024-03-18T15:33:41.996414+0000","last_unstale":"2024-03-18T15:33:45.097620+0000","last_undegraded":"2024-03-18T15:33:45.097620+0000","last_fullsized":"2024-03-18T15:33:45.097620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:48:24.038250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.096753+0000","last_change":"2024-03-18T15:33:43.009392+0000","last_active":"2024-03-18T15:33:45.096753+0000","last_peered":"2024-03-18T15:33:45.096753+0000","last_clean":"2024-03-18T15:33:45.096753+0000","last_became_active":"2024-03-18T15:33:42.997447+0000","last_became_peered":"2024-03-18T15:33:42.997447+0000","last_unstale":"2024-03-18T15:33:45.096753+0000","last_undegraded":"2024-03-18T15:33:45.096753+0000","last_fullsized":"2024-03-18T15:33:45.096753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:35:43.006937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.096207+0000","last_change":"2024-03-18T15:33:42.002322+0000","last_active":"2024-03-18T15:33:45.096207+0000","last_peered":"2024-03-18T15:33:45.096207+0000","last_clean":"2024-03-18T15:33:45.096207+0000","last_became_active":"2024-03-18T15:33:42.001646+0000","last_became_peered":"2024-03-18T15:33:42.001646+0000","last_unstale":"2024-03-18T15:33:45.096207+0000","last_undegraded":"2024-03-18T15:33:45.096207+0000","last_fullsized":"2024-03-18T15:33:45.096207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:39:46.475100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.995243+0000","last_change":"2024-03-18T15:33:42.998180+0000","last_active":"2024-03-18T15:33:43.995243+0000","last_peered":"2024-03-18T15:33:43.995243+0000","last_clean":"2024-03-18T15:33:43.995243+0000","last_became_active":"2024-03-18T15:33:42.997429+0000","last_became_peered":"2024-03-18T15:33:42.997429+0000","last_unstale":"2024-03-18T15:33:43.995243+0000","last_undegraded":"2024-03-18T15:33:43.995243+0000","last_fullsized":"2024-03-18T15:33:43.995243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:03:19.737667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.002488+0000","last_change":"2024-03-18T15:33:42.005707+0000","last_active":"2024-03-18T15:33:44.002488+0000","last_peered":"2024-03-18T15:33:44.002488+0000","last_clean":"2024-03-18T15:33:44.002488+0000","last_became_active":"2024-03-18T15:33:42.004925+0000","last_became_peered":"2024-03-18T15:33:42.004925+0000","last_unstale":"2024-03-18T15:33:44.002488+0000","last_undegraded":"2024-03-18T15:33:44.002488+0000","last_fullsized":"2024-03-18T15:33:44.002488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:39:46.475100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.096274+0000","last_change":"2024-03-18T15:33:43.012260+0000","last_active":"2024-03-18T15:33:45.096274+0000","last_peered":"2024-03-18T15:33:45.096274+0000","last_clean":"2024-03-18T15:33:45.096274+0000","last_became_active":"2024-03-18T15:33:43.011884+0000","last_became_peered":"2024-03-18T15:33:43.011884+0000","last_unstale":"2024-03-18T15:33:45.096274+0000","last_undegraded":"2024-03-18T15:33:45.096274+0000","last_fullsized":"2024-03-18T15:33:45.096274+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:33:26.178317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.096423+0000","last_change":"2024-03-18T15:33:42.006507+0000","last_active":"2024-03-18T15:33:45.096423+0000","last_peered":"2024-03-18T15:33:45.096423+0000","last_clean":"2024-03-18T15:33:45.096423+0000","last_became_active":"2024-03-18T15:33:42.005178+0000","last_became_peered":"2024-03-18T15:33:42.005178+0000","last_unstale":"2024-03-18T15:33:45.096423+0000","last_undegraded":"2024-03-18T15:33:45.096423+0000","last_fullsized":"2024-03-18T15:33:45.096423+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:09:14.869832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.097557+0000","last_change":"2024-03-18T15:33:43.021210+0000","last_active":"2024-03-18T15:33:45.097557+0000","last_peered":"2024-03-18T15:33:45.097557+0000","last_clean":"2024-03-18T15:33:45.097557+0000","last_became_active":"2024-03-18T15:33:43.020819+0000","last_became_peered":"2024-03-18T15:33:43.020819+0000","last_unstale":"2024-03-18T15:33:45.097557+0000","last_undegraded":"2024-03-18T15:33:45.097557+0000","last_fullsized":"2024-03-18T15:33:45.097557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:50:28.968807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.996129+0000","last_change":"2024-03-18T15:33:42.000763+0000","last_active":"2024-03-18T15:33:43.996129+0000","last_peered":"2024-03-18T15:33:43.996129+0000","last_clean":"2024-03-18T15:33:43.996129+0000","last_became_active":"2024-03-18T15:33:42.000392+0000","last_became_peered":"2024-03-18T15:33:42.000392+0000","last_unstale":"2024-03-18T15:33:43.996129+0000","last_undegraded":"2024-03-18T15:33:43.996129+0000","last_fullsized":"2024-03-18T15:33:43.996129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:43:41.408584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.995096+0000","last_change":"2024-03-18T15:33:43.003710+0000","last_active":"2024-03-18T15:33:43.995096+0000","last_peered":"2024-03-18T15:33:43.995096+0000","last_clean":"2024-03-18T15:33:43.995096+0000","last_became_active":"2024-03-18T15:33:43.003388+0000","last_became_peered":"2024-03-18T15:33:43.003388+0000","last_unstale":"2024-03-18T15:33:43.995096+0000","last_undegraded":"2024-03-18T15:33:43.995096+0000","last_fullsized":"2024-03-18T15:33:43.995096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:33:12.492530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.002393+0000","last_change":"2024-03-18T15:33:41.993113+0000","last_active":"2024-03-18T15:33:44.002393+0000","last_peered":"2024-03-18T15:33:44.002393+0000","last_clean":"2024-03-18T15:33:44.002393+0000","last_became_active":"2024-03-18T15:33:41.992459+0000","last_became_peered":"2024-03-18T15:33:41.992459+0000","last_unstale":"2024-03-18T15:33:44.002393+0000","last_undegraded":"2024-03-18T15:33:44.002393+0000","last_fullsized":"2024-03-18T15:33:44.002393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:26:02.875362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.993666+0000","last_change":"2024-03-18T15:33:43.010684+0000","last_active":"2024-03-18T15:33:43.993666+0000","last_peered":"2024-03-18T15:33:43.993666+0000","last_clean":"2024-03-18T15:33:43.993666+0000","last_became_active":"2024-03-18T15:33:43.010021+0000","last_became_peered":"2024-03-18T15:33:43.010021+0000","last_unstale":"2024-03-18T15:33:43.993666+0000","last_undegraded":"2024-03-18T15:33:43.993666+0000","last_fullsized":"2024-03-18T15:33:43.993666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:19:11.577023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.096901+0000","last_change":"2024-03-18T15:33:41.991859+0000","last_active":"2024-03-18T15:33:45.096901+0000","last_peered":"2024-03-18T15:33:45.096901+0000","last_clean":"2024-03-18T15:33:45.096901+0000","last_became_active":"2024-03-18T15:33:41.987429+0000","last_became_peered":"2024-03-18T15:33:41.987429+0000","last_unstale":"2024-03-18T15:33:45.096901+0000","last_undegraded":"2024-03-18T15:33:45.096901+0000","last_fullsized":"2024-03-18T15:33:45.096901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:41:48.671691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.096587+0000","last_change":"2024-03-18T15:33:43.014886+0000","last_active":"2024-03-18T15:33:45.096587+0000","last_peered":"2024-03-18T15:33:45.096587+0000","last_clean":"2024-03-18T15:33:45.096587+0000","last_became_active":"2024-03-18T15:33:43.014430+0000","last_became_peered":"2024-03-18T15:33:43.014430+0000","last_unstale":"2024-03-18T15:33:45.096587+0000","last_undegraded":"2024-03-18T15:33:45.096587+0000","last_fullsized":"2024-03-18T15:33:45.096587+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:59:23.040679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.095989+0000","last_change":"2024-03-18T15:33:42.000527+0000","last_active":"2024-03-18T15:33:45.095989+0000","last_peered":"2024-03-18T15:33:45.095989+0000","last_clean":"2024-03-18T15:33:45.095989+0000","last_became_active":"2024-03-18T15:33:41.999663+0000","last_became_peered":"2024-03-18T15:33:41.999663+0000","last_unstale":"2024-03-18T15:33:45.095989+0000","last_undegraded":"2024-03-18T15:33:45.095989+0000","last_fullsized":"2024-03-18T15:33:45.095989+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:26:02.875362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.994248+0000","last_change":"2024-03-18T15:33:42.997482+0000","last_active":"2024-03-18T15:33:43.994248+0000","last_peered":"2024-03-18T15:33:43.994248+0000","last_clean":"2024-03-18T15:33:43.994248+0000","last_became_active":"2024-03-18T15:33:42.993674+0000","last_became_peered":"2024-03-18T15:33:42.993674+0000","last_unstale":"2024-03-18T15:33:43.994248+0000","last_undegraded":"2024-03-18T15:33:43.994248+0000","last_fullsized":"2024-03-18T15:33:43.994248+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:04:58.261410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.001972+0000","last_change":"2024-03-18T15:33:41.990751+0000","last_active":"2024-03-18T15:33:44.001972+0000","last_peered":"2024-03-18T15:33:44.001972+0000","last_clean":"2024-03-18T15:33:44.001972+0000","last_became_active":"2024-03-18T15:33:41.989537+0000","last_became_peered":"2024-03-18T15:33:41.989537+0000","last_unstale":"2024-03-18T15:33:44.001972+0000","last_undegraded":"2024-03-18T15:33:44.001972+0000","last_fullsized":"2024-03-18T15:33:44.001972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:09:14.869832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.002581+0000","last_change":"2024-03-18T15:33:43.000356+0000","last_active":"2024-03-18T15:33:44.002581+0000","last_peered":"2024-03-18T15:33:44.002581+0000","last_clean":"2024-03-18T15:33:44.002581+0000","last_became_active":"2024-03-18T15:33:43.000031+0000","last_became_peered":"2024-03-18T15:33:43.000031+0000","last_unstale":"2024-03-18T15:33:44.002581+0000","last_undegraded":"2024-03-18T15:33:44.002581+0000","last_fullsized":"2024-03-18T15:33:44.002581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:24:37.779979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.869203+0000","last_change":"2024-03-18T15:33:41.976955+0000","last_active":"2024-03-18T15:33:44.869203+0000","last_peered":"2024-03-18T15:33:44.869203+0000","last_clean":"2024-03-18T15:33:44.869203+0000","last_became_active":"2024-03-18T15:33:41.976330+0000","last_became_peered":"2024-03-18T15:33:41.976330+0000","last_unstale":"2024-03-18T15:33:44.869203+0000","last_undegraded":"2024-03-18T15:33:44.869203+0000","last_fullsized":"2024-03-18T15:33:44.869203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:38:45.057586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.993542+0000","last_change":"2024-03-18T15:33:42.994227+0000","last_active":"2024-03-18T15:33:43.993542+0000","last_peered":"2024-03-18T15:33:43.993542+0000","last_clean":"2024-03-18T15:33:43.993542+0000","last_became_active":"2024-03-18T15:33:42.993870+0000","last_became_peered":"2024-03-18T15:33:42.993870+0000","last_unstale":"2024-03-18T15:33:43.993542+0000","last_undegraded":"2024-03-18T15:33:43.993542+0000","last_fullsized":"2024-03-18T15:33:43.993542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:29:15.496652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.994431+0000","last_change":"2024-03-18T15:33:41.987050+0000","last_active":"2024-03-18T15:33:43.994431+0000","last_peered":"2024-03-18T15:33:43.994431+0000","last_clean":"2024-03-18T15:33:43.994431+0000","last_became_active":"2024-03-18T15:33:41.986209+0000","last_became_peered":"2024-03-18T15:33:41.986209+0000","last_unstale":"2024-03-18T15:33:43.994431+0000","last_undegraded":"2024-03-18T15:33:43.994431+0000","last_fullsized":"2024-03-18T15:33:43.994431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:41:28.732114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.995320+0000","last_change":"2024-03-18T15:33:42.998067+0000","last_active":"2024-03-18T15:33:43.995320+0000","last_peered":"2024-03-18T15:33:43.995320+0000","last_clean":"2024-03-18T15:33:43.995320+0000","last_became_active":"2024-03-18T15:33:42.997755+0000","last_became_peered":"2024-03-18T15:33:42.997755+0000","last_unstale":"2024-03-18T15:33:43.995320+0000","last_undegraded":"2024-03-18T15:33:43.995320+0000","last_fullsized":"2024-03-18T15:33:43.995320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:10:01.824296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.869684+0000","last_change":"2024-03-18T15:33:41.987009+0000","last_active":"2024-03-18T15:33:44.869684+0000","last_peered":"2024-03-18T15:33:44.869684+0000","last_clean":"2024-03-18T15:33:44.869684+0000","last_became_active":"2024-03-18T15:33:41.986376+0000","last_became_peered":"2024-03-18T15:33:41.986376+0000","last_unstale":"2024-03-18T15:33:44.869684+0000","last_undegraded":"2024-03-18T15:33:44.869684+0000","last_fullsized":"2024-03-18T15:33:44.869684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:26:02.875362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.994960+0000","last_change":"2024-03-18T15:33:43.007612+0000","last_active":"2024-03-18T15:33:43.994960+0000","last_peered":"2024-03-18T15:33:43.994960+0000","last_clean":"2024-03-18T15:33:43.994960+0000","last_became_active":"2024-03-18T15:33:43.006950+0000","last_became_peered":"2024-03-18T15:33:43.006950+0000","last_unstale":"2024-03-18T15:33:43.994960+0000","last_undegraded":"2024-03-18T15:33:43.994960+0000","last_fullsized":"2024-03-18T15:33:43.994960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:52:46.912729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.996448+0000","last_change":"2024-03-18T15:33:41.990256+0000","last_active":"2024-03-18T15:33:43.996448+0000","last_peered":"2024-03-18T15:33:43.996448+0000","last_clean":"2024-03-18T15:33:43.996448+0000","last_became_active":"2024-03-18T15:33:41.989715+0000","last_became_peered":"2024-03-18T15:33:41.989715+0000","last_unstale":"2024-03-18T15:33:43.996448+0000","last_undegraded":"2024-03-18T15:33:43.996448+0000","last_fullsized":"2024-03-18T15:33:43.996448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:23:20.085212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.096441+0000","last_change":"2024-03-18T15:33:43.022519+0000","last_active":"2024-03-18T15:33:45.096441+0000","last_peered":"2024-03-18T15:33:45.096441+0000","last_clean":"2024-03-18T15:33:45.096441+0000","last_became_active":"2024-03-18T15:33:43.022081+0000","last_became_peered":"2024-03-18T15:33:43.022081+0000","last_unstale":"2024-03-18T15:33:45.096441+0000","last_undegraded":"2024-03-18T15:33:45.096441+0000","last_fullsized":"2024-03-18T15:33:45.096441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:09:14.711960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.869440+0000","last_change":"2024-03-18T15:33:42.001343+0000","last_active":"2024-03-18T15:33:44.869440+0000","last_peered":"2024-03-18T15:33:44.869440+0000","last_clean":"2024-03-18T15:33:44.869440+0000","last_became_active":"2024-03-18T15:33:42.000948+0000","last_became_peered":"2024-03-18T15:33:42.000948+0000","last_unstale":"2024-03-18T15:33:44.869440+0000","last_undegraded":"2024-03-18T15:33:44.869440+0000","last_fullsized":"2024-03-18T15:33:44.869440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:47:53.438385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.993920+0000","last_change":"2024-03-18T15:33:43.015405+0000","last_active":"2024-03-18T15:33:43.993920+0000","last_peered":"2024-03-18T15:33:43.993920+0000","last_clean":"2024-03-18T15:33:43.993920+0000","last_became_active":"2024-03-18T15:33:43.015094+0000","last_became_peered":"2024-03-18T15:33:43.015094+0000","last_unstale":"2024-03-18T15:33:43.993920+0000","last_undegraded":"2024-03-18T15:33:43.993920+0000","last_fullsized":"2024-03-18T15:33:43.993920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:30:16.022611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.993319+0000","last_change":"2024-03-18T15:33:30.766315+0000","last_active":"2024-03-18T15:33:43.993319+0000","last_peered":"2024-03-18T15:33:43.993319+0000","last_clean":"2024-03-18T15:33:43.993319+0000","last_became_active":"2024-03-18T15:33:30.763939+0000","last_became_peered":"2024-03-18T15:33:30.763939+0000","last_unstale":"2024-03-18T15:33:43.993319+0000","last_undegraded":"2024-03-18T15:33:43.993319+0000","last_fullsized":"2024-03-18T15:33:43.993319+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:29.229911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:29.229911+0000","last_clean_scrub_stamp":"2024-03-18T15:33:29.229911+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:10:21.120371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.095640+0000","last_change":"2024-03-18T15:33:41.998201+0000","last_active":"2024-03-18T15:33:45.095640+0000","last_peered":"2024-03-18T15:33:45.095640+0000","last_clean":"2024-03-18T15:33:45.095640+0000","last_became_active":"2024-03-18T15:33:41.997079+0000","last_became_peered":"2024-03-18T15:33:41.997079+0000","last_unstale":"2024-03-18T15:33:45.095640+0000","last_undegraded":"2024-03-18T15:33:45.095640+0000","last_fullsized":"2024-03-18T15:33:45.095640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:38:45.057586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.097810+0000","last_change":"2024-03-18T15:33:43.019262+0000","last_active":"2024-03-18T15:33:45.097810+0000","last_peered":"2024-03-18T15:33:45.097810+0000","last_clean":"2024-03-18T15:33:45.097810+0000","last_became_active":"2024-03-18T15:33:43.018918+0000","last_became_peered":"2024-03-18T15:33:43.018918+0000","last_unstale":"2024-03-18T15:33:45.097810+0000","last_undegraded":"2024-03-18T15:33:45.097810+0000","last_fullsized":"2024-03-18T15:33:45.097810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:16:44.575392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.993340+0000","last_change":"2024-03-18T15:33:42.003947+0000","last_active":"2024-03-18T15:33:43.993340+0000","last_peered":"2024-03-18T15:33:43.993340+0000","last_clean":"2024-03-18T15:33:43.993340+0000","last_became_active":"2024-03-18T15:33:42.002250+0000","last_became_peered":"2024-03-18T15:33:42.002250+0000","last_unstale":"2024-03-18T15:33:43.993340+0000","last_undegraded":"2024-03-18T15:33:43.993340+0000","last_fullsized":"2024-03-18T15:33:43.993340+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:07:04.089081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.096070+0000","last_change":"2024-03-18T15:33:43.015094+0000","last_active":"2024-03-18T15:33:45.096070+0000","last_peered":"2024-03-18T15:33:45.096070+0000","last_clean":"2024-03-18T15:33:45.096070+0000","last_became_active":"2024-03-18T15:33:43.014596+0000","last_became_peered":"2024-03-18T15:33:43.014596+0000","last_unstale":"2024-03-18T15:33:45.096070+0000","last_undegraded":"2024-03-18T15:33:45.096070+0000","last_fullsized":"2024-03-18T15:33:45.096070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:31:24.037103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.869444+0000","last_change":"2024-03-18T15:33:41.991149+0000","last_active":"2024-03-18T15:33:44.869444+0000","last_peered":"2024-03-18T15:33:44.869444+0000","last_clean":"2024-03-18T15:33:44.869444+0000","last_became_active":"2024-03-18T15:33:41.990059+0000","last_became_peered":"2024-03-18T15:33:41.990059+0000","last_unstale":"2024-03-18T15:33:44.869444+0000","last_undegraded":"2024-03-18T15:33:44.869444+0000","last_fullsized":"2024-03-18T15:33:44.869444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:09:14.869832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.995704+0000","last_change":"2024-03-18T15:33:43.000746+0000","last_active":"2024-03-18T15:33:43.995704+0000","last_peered":"2024-03-18T15:33:43.995704+0000","last_clean":"2024-03-18T15:33:43.995704+0000","last_became_active":"2024-03-18T15:33:42.999935+0000","last_became_peered":"2024-03-18T15:33:42.999935+0000","last_unstale":"2024-03-18T15:33:43.995704+0000","last_undegraded":"2024-03-18T15:33:43.995704+0000","last_fullsized":"2024-03-18T15:33:43.995704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:11:30.652938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.096162+0000","last_change":"2024-03-18T15:33:42.002409+0000","last_active":"2024-03-18T15:33:45.096162+0000","last_peered":"2024-03-18T15:33:45.096162+0000","last_clean":"2024-03-18T15:33:45.096162+0000","last_became_active":"2024-03-18T15:33:42.000616+0000","last_became_peered":"2024-03-18T15:33:42.000616+0000","last_unstale":"2024-03-18T15:33:45.096162+0000","last_undegraded":"2024-03-18T15:33:45.096162+0000","last_fullsized":"2024-03-18T15:33:45.096162+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:47:53.438385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.994828+0000","last_change":"2024-03-18T15:33:43.005420+0000","last_active":"2024-03-18T15:33:43.994828+0000","last_peered":"2024-03-18T15:33:43.994828+0000","last_clean":"2024-03-18T15:33:43.994828+0000","last_became_active":"2024-03-18T15:33:43.004488+0000","last_became_peered":"2024-03-18T15:33:43.004488+0000","last_unstale":"2024-03-18T15:33:43.994828+0000","last_undegraded":"2024-03-18T15:33:43.994828+0000","last_fullsized":"2024-03-18T15:33:43.994828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:35:46.663732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.996031+0000","last_change":"2024-03-18T15:33:41.990472+0000","last_active":"2024-03-18T15:33:43.996031+0000","last_peered":"2024-03-18T15:33:43.996031+0000","last_clean":"2024-03-18T15:33:43.996031+0000","last_became_active":"2024-03-18T15:33:41.989782+0000","last_became_peered":"2024-03-18T15:33:41.989782+0000","last_unstale":"2024-03-18T15:33:43.996031+0000","last_undegraded":"2024-03-18T15:33:43.996031+0000","last_fullsized":"2024-03-18T15:33:43.996031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:35:45.651780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.096233+0000","last_change":"2024-03-18T15:33:43.013011+0000","last_active":"2024-03-18T15:33:45.096233+0000","last_peered":"2024-03-18T15:33:45.096233+0000","last_clean":"2024-03-18T15:33:45.096233+0000","last_became_active":"2024-03-18T15:33:43.012334+0000","last_became_peered":"2024-03-18T15:33:43.012334+0000","last_unstale":"2024-03-18T15:33:45.096233+0000","last_undegraded":"2024-03-18T15:33:45.096233+0000","last_fullsized":"2024-03-18T15:33:45.096233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:43:15.113425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.994085+0000","last_change":"2024-03-18T15:33:42.011960+0000","last_active":"2024-03-18T15:33:43.994085+0000","last_peered":"2024-03-18T15:33:43.994085+0000","last_clean":"2024-03-18T15:33:43.994085+0000","last_became_active":"2024-03-18T15:33:42.010760+0000","last_became_peered":"2024-03-18T15:33:42.010760+0000","last_unstale":"2024-03-18T15:33:43.994085+0000","last_undegraded":"2024-03-18T15:33:43.994085+0000","last_fullsized":"2024-03-18T15:33:43.994085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:29:33.713515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.996001+0000","last_change":"2024-03-18T15:33:42.985474+0000","last_active":"2024-03-18T15:33:43.996001+0000","last_peered":"2024-03-18T15:33:43.996001+0000","last_clean":"2024-03-18T15:33:43.996001+0000","last_became_active":"2024-03-18T15:33:42.984659+0000","last_became_peered":"2024-03-18T15:33:42.984659+0000","last_unstale":"2024-03-18T15:33:43.996001+0000","last_undegraded":"2024-03-18T15:33:43.996001+0000","last_fullsized":"2024-03-18T15:33:43.996001+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:38:08.174152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.996599+0000","last_change":"2024-03-18T15:33:41.994297+0000","last_active":"2024-03-18T15:33:43.996599+0000","last_peered":"2024-03-18T15:33:43.996599+0000","last_clean":"2024-03-18T15:33:43.996599+0000","last_became_active":"2024-03-18T15:33:41.993906+0000","last_became_peered":"2024-03-18T15:33:41.993906+0000","last_unstale":"2024-03-18T15:33:43.996599+0000","last_undegraded":"2024-03-18T15:33:43.996599+0000","last_fullsized":"2024-03-18T15:33:43.996599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:07:04.089081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.099121+0000","last_change":"2024-03-18T15:33:43.018039+0000","last_active":"2024-03-18T15:33:45.099121+0000","last_peered":"2024-03-18T15:33:45.099121+0000","last_clean":"2024-03-18T15:33:45.099121+0000","last_became_active":"2024-03-18T15:33:43.017680+0000","last_became_peered":"2024-03-18T15:33:43.017680+0000","last_unstale":"2024-03-18T15:33:45.099121+0000","last_undegraded":"2024-03-18T15:33:45.099121+0000","last_fullsized":"2024-03-18T15:33:45.099121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:29:15.496652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.002591+0000","last_change":"2024-03-18T15:33:42.009920+0000","last_active":"2024-03-18T15:33:44.002591+0000","last_peered":"2024-03-18T15:33:44.002591+0000","last_clean":"2024-03-18T15:33:44.002591+0000","last_became_active":"2024-03-18T15:33:42.009394+0000","last_became_peered":"2024-03-18T15:33:42.009394+0000","last_unstale":"2024-03-18T15:33:44.002591+0000","last_undegraded":"2024-03-18T15:33:44.002591+0000","last_fullsized":"2024-03-18T15:33:44.002591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:27:05.573887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.097310+0000","last_change":"2024-03-18T15:33:42.999878+0000","last_active":"2024-03-18T15:33:45.097310+0000","last_peered":"2024-03-18T15:33:45.097310+0000","last_clean":"2024-03-18T15:33:45.097310+0000","last_became_active":"2024-03-18T15:33:42.999140+0000","last_became_peered":"2024-03-18T15:33:42.999140+0000","last_unstale":"2024-03-18T15:33:45.097310+0000","last_undegraded":"2024-03-18T15:33:45.097310+0000","last_fullsized":"2024-03-18T15:33:45.097310+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:28:55.955807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.993177+0000","last_change":"2024-03-18T15:33:42.009110+0000","last_active":"2024-03-18T15:33:43.993177+0000","last_peered":"2024-03-18T15:33:43.993177+0000","last_clean":"2024-03-18T15:33:43.993177+0000","last_became_active":"2024-03-18T15:33:42.008110+0000","last_became_peered":"2024-03-18T15:33:42.008110+0000","last_unstale":"2024-03-18T15:33:43.993177+0000","last_undegraded":"2024-03-18T15:33:43.993177+0000","last_fullsized":"2024-03-18T15:33:43.993177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:23:20.085212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.994867+0000","last_change":"2024-03-18T15:33:43.006171+0000","last_active":"2024-03-18T15:33:43.994867+0000","last_peered":"2024-03-18T15:33:43.994867+0000","last_clean":"2024-03-18T15:33:43.994867+0000","last_became_active":"2024-03-18T15:33:43.005851+0000","last_became_peered":"2024-03-18T15:33:43.005851+0000","last_unstale":"2024-03-18T15:33:43.994867+0000","last_undegraded":"2024-03-18T15:33:43.994867+0000","last_fullsized":"2024-03-18T15:33:43.994867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:23:21.097164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.995361+0000","last_change":"2024-03-18T15:33:41.997380+0000","last_active":"2024-03-18T15:33:43.995361+0000","last_peered":"2024-03-18T15:33:43.995361+0000","last_clean":"2024-03-18T15:33:43.995361+0000","last_became_active":"2024-03-18T15:33:41.997101+0000","last_became_peered":"2024-03-18T15:33:41.997101+0000","last_unstale":"2024-03-18T15:33:43.995361+0000","last_undegraded":"2024-03-18T15:33:43.995361+0000","last_fullsized":"2024-03-18T15:33:43.995361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:33:11.480578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.097164+0000","last_change":"2024-03-18T15:33:43.012575+0000","last_active":"2024-03-18T15:33:45.097164+0000","last_peered":"2024-03-18T15:33:45.097164+0000","last_clean":"2024-03-18T15:33:45.097164+0000","last_became_active":"2024-03-18T15:33:42.996972+0000","last_became_peered":"2024-03-18T15:33:42.996972+0000","last_unstale":"2024-03-18T15:33:45.097164+0000","last_undegraded":"2024-03-18T15:33:45.097164+0000","last_fullsized":"2024-03-18T15:33:45.097164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:08:08.722076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.996259+0000","last_change":"2024-03-18T15:33:41.990440+0000","last_active":"2024-03-18T15:33:43.996259+0000","last_peered":"2024-03-18T15:33:43.996259+0000","last_clean":"2024-03-18T15:33:43.996259+0000","last_became_active":"2024-03-18T15:33:41.989808+0000","last_became_peered":"2024-03-18T15:33:41.989808+0000","last_unstale":"2024-03-18T15:33:43.996259+0000","last_undegraded":"2024-03-18T15:33:43.996259+0000","last_fullsized":"2024-03-18T15:33:43.996259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:14:42.231258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.098120+0000","last_change":"2024-03-18T15:33:43.012652+0000","last_active":"2024-03-18T15:33:45.098120+0000","last_peered":"2024-03-18T15:33:45.098120+0000","last_clean":"2024-03-18T15:33:45.098120+0000","last_became_active":"2024-03-18T15:33:43.010391+0000","last_became_peered":"2024-03-18T15:33:43.010391+0000","last_unstale":"2024-03-18T15:33:45.098120+0000","last_undegraded":"2024-03-18T15:33:45.098120+0000","last_fullsized":"2024-03-18T15:33:45.098120+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:38:08.174152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.994505+0000","last_change":"2024-03-18T15:33:42.013549+0000","last_active":"2024-03-18T15:33:43.994505+0000","last_peered":"2024-03-18T15:33:43.994505+0000","last_clean":"2024-03-18T15:33:43.994505+0000","last_became_active":"2024-03-18T15:33:42.012707+0000","last_became_peered":"2024-03-18T15:33:42.012707+0000","last_unstale":"2024-03-18T15:33:43.994505+0000","last_undegraded":"2024-03-18T15:33:43.994505+0000","last_fullsized":"2024-03-18T15:33:43.994505+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:14:42.231258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:44.002049+0000","last_change":"2024-03-18T15:33:43.003144+0000","last_active":"2024-03-18T15:33:44.002049+0000","last_peered":"2024-03-18T15:33:44.002049+0000","last_clean":"2024-03-18T15:33:44.002049+0000","last_became_active":"2024-03-18T15:33:43.002771+0000","last_became_peered":"2024-03-18T15:33:43.002771+0000","last_unstale":"2024-03-18T15:33:44.002049+0000","last_undegraded":"2024-03-18T15:33:44.002049+0000","last_fullsized":"2024-03-18T15:33:44.002049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:50:43.021257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.096229+0000","last_change":"2024-03-18T15:33:43.014403+0000","last_active":"2024-03-18T15:33:45.096229+0000","last_peered":"2024-03-18T15:33:45.096229+0000","last_clean":"2024-03-18T15:33:45.096229+0000","last_became_active":"2024-03-18T15:33:43.012667+0000","last_became_peered":"2024-03-18T15:33:43.012667+0000","last_unstale":"2024-03-18T15:33:45.096229+0000","last_undegraded":"2024-03-18T15:33:45.096229+0000","last_fullsized":"2024-03-18T15:33:45.096229+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:50:44.996218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.993602+0000","last_change":"2024-03-18T15:33:42.009430+0000","last_active":"2024-03-18T15:33:43.993602+0000","last_peered":"2024-03-18T15:33:43.993602+0000","last_clean":"2024-03-18T15:33:43.993602+0000","last_became_active":"2024-03-18T15:33:42.008331+0000","last_became_peered":"2024-03-18T15:33:42.008331+0000","last_unstale":"2024-03-18T15:33:43.993602+0000","last_undegraded":"2024-03-18T15:33:43.993602+0000","last_fullsized":"2024-03-18T15:33:43.993602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:33:11.480578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.096113+0000","last_change":"2024-03-18T15:33:43.017670+0000","last_active":"2024-03-18T15:33:45.096113+0000","last_peered":"2024-03-18T15:33:45.096113+0000","last_clean":"2024-03-18T15:33:45.096113+0000","last_became_active":"2024-03-18T15:33:43.017297+0000","last_became_peered":"2024-03-18T15:33:43.017297+0000","last_unstale":"2024-03-18T15:33:45.096113+0000","last_undegraded":"2024-03-18T15:33:45.096113+0000","last_fullsized":"2024-03-18T15:33:45.096113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:30:50.311074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:43.993805+0000","last_change":"2024-03-18T15:33:42.009132+0000","last_active":"2024-03-18T15:33:43.993805+0000","last_peered":"2024-03-18T15:33:43.993805+0000","last_clean":"2024-03-18T15:33:43.993805+0000","last_became_active":"2024-03-18T15:33:42.008343+0000","last_became_peered":"2024-03-18T15:33:42.008343+0000","last_unstale":"2024-03-18T15:33:43.993805+0000","last_undegraded":"2024-03-18T15:33:43.993805+0000","last_fullsized":"2024-03-18T15:33:43.993805+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T15:35:22.723629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.097441+0000","last_change":"2024-03-18T15:33:43.021277+0000","last_active":"2024-03-18T15:33:45.097441+0000","last_peered":"2024-03-18T15:33:45.097441+0000","last_clean":"2024-03-18T15:33:45.097441+0000","last_became_active":"2024-03-18T15:33:43.020857+0000","last_became_peered":"2024-03-18T15:33:43.020857+0000","last_unstale":"2024-03-18T15:33:45.097441+0000","last_undegraded":"2024-03-18T15:33:45.097441+0000","last_fullsized":"2024-03-18T15:33:45.097441+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:24:37.779979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.098590+0000","last_change":"2024-03-18T15:33:41.992048+0000","last_active":"2024-03-18T15:33:45.098590+0000","last_peered":"2024-03-18T15:33:45.098590+0000","last_clean":"2024-03-18T15:33:45.098590+0000","last_became_active":"2024-03-18T15:33:41.989055+0000","last_became_peered":"2024-03-18T15:33:41.989055+0000","last_unstale":"2024-03-18T15:33:45.098590+0000","last_undegraded":"2024-03-18T15:33:45.098590+0000","last_fullsized":"2024-03-18T15:33:45.098590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:46:33.341037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.098751+0000","last_change":"2024-03-18T15:33:42.996569+0000","last_active":"2024-03-18T15:33:45.098751+0000","last_peered":"2024-03-18T15:33:45.098751+0000","last_clean":"2024-03-18T15:33:45.098751+0000","last_became_active":"2024-03-18T15:33:42.996234+0000","last_became_peered":"2024-03-18T15:33:42.996234+0000","last_unstale":"2024-03-18T15:33:45.098751+0000","last_undegraded":"2024-03-18T15:33:45.098751+0000","last_fullsized":"2024-03-18T15:33:45.098751+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:41.952065+0000","last_clean_scrub_stamp":"2024-03-18T15:33:41.952065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:29:18.287234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-18T15:33:45.097738+0000","last_change":"2024-03-18T15:33:42.010405+0000","last_active":"2024-03-18T15:33:45.097738+0000","last_peered":"2024-03-18T15:33:45.097738+0000","last_clean":"2024-03-18T15:33:45.097738+0000","last_became_active":"2024-03-18T15:33:42.009883+0000","last_became_peered":"2024-03-18T15:33:42.009883+0000","last_unstale":"2024-03-18T15:33:45.097738+0000","last_undegraded":"2024-03-18T15:33:45.097738+0000","last_fullsized":"2024-03-18T15:33:45.097738+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:40.940113+0000","last_clean_scrub_stamp":"2024-03-18T15:33:40.940113+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:10:06.802810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":221184,"data_stored":75067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":73545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":806912,"data_stored":664405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542152,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":73581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35556,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336284,"statfs":{"total":96636764160,"available":96600354816,"internally_reserved":0,"allocated":221184,"data_stored":75067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35544,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336296,"statfs":{"total":96636764160,"available":96600367104,"internally_reserved":0,"allocated":208896,"data_stored":73455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":73615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36136,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335704,"statfs":{"total":96636764160,"available":96599760896,"internally_reserved":0,"allocated":815104,"data_stored":664461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-18T15:33:49.994 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-18T15:33:49.994 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-18T15:33:49.994 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-03-18T15:33:49.994 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-18T15:33:50.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.149+0000 7fedd7b68640 1 Processor -- start 2024-03-18T15:33:50.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.153+0000 7fedd7b68640 1 -- start start 2024-03-18T15:33:50.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.153+0000 7fedd7b68640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fedd00b5ee0 0x7fedd00af000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:50.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.153+0000 7fedd7b68640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fedd0148390 0x7fedd014eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:50.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.153+0000 7fedd7b68640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fedd00b62c0 con 0x7fedd00b4630 2024-03-18T15:33:50.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.153+0000 7fedd7b68640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fedd00b0c90 con 0x7fedd00b5ee0 2024-03-18T15:33:50.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.153+0000 7fedd7b68640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fedd00b4ac0 con 0x7fedd0148390 2024-03-18T15:33:50.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.153+0000 7fedd5103640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fedd00b4630 0x7fedd0147c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:57322/0 (socket says 172.21.15.5:57322) 2024-03-18T15:33:50.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.153+0000 7fedd5904640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fedd00b5ee0 0x7fedd00af000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:50.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.153+0000 7fedd5103640 1 -- 172.21.15.5:0/2232647707 learned_addr learned my addr 172.21.15.5:0/2232647707 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:33:50.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.153+0000 7fedd6105640 1 --2- 172.21.15.5:0/2232647707 >> v2:172.21.15.80:3300/0 conn(0x7fedd0148390 0x7fedd014eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:50.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.153+0000 7fedd4902640 1 -- 172.21.15.5:0/2232647707 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1566851024 0 0) 0x7fedd00b62c0 con 0x7fedd00b4630 2024-03-18T15:33:50.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.153+0000 7fedd4902640 1 -- 172.21.15.5:0/2232647707 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fedbc005740 con 0x7fedd00b4630 2024-03-18T15:33:50.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.153+0000 7fedd5904640 1 -- 172.21.15.5:0/2232647707 >> v2:172.21.15.80:3300/0 conn(0x7fedd0148390 msgr2=0x7fedd014eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:50.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.153+0000 7fedd5904640 1 --2- 172.21.15.5:0/2232647707 >> v2:172.21.15.80:3300/0 conn(0x7fedd0148390 0x7fedd014eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:50.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.153+0000 7fedd5904640 1 -- 172.21.15.5:0/2232647707 >> v1:172.21.15.5:6789/0 conn(0x7fedd00b4630 legacy=0x7fedd0147c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:50.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.153+0000 7fedd5904640 1 -- 172.21.15.5:0/2232647707 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedd0072610 con 0x7fedd00b5ee0 2024-03-18T15:33:50.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.153+0000 7fedd5904640 1 --2- 172.21.15.5:0/2232647707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedd00b5ee0 0x7fedd00af000 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fedd00b0660 tx=0x7fedcc01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ead0dfc442e5bda7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:50.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.153+0000 7fedd4902640 1 -- 172.21.15.5:0/2232647707 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc0b0070 con 0x7fedd00b5ee0 2024-03-18T15:33:50.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.153+0000 7fedd4902640 1 -- 172.21.15.5:0/2232647707 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedcc0ab600 con 0x7fedd00b5ee0 2024-03-18T15:33:50.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.153+0000 7fedd4902640 1 -- 172.21.15.5:0/2232647707 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedcc0ab9d0 con 0x7fedd00b5ee0 2024-03-18T15:33:50.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.157+0000 7fedd7b68640 1 -- 172.21.15.5:0/2232647707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedd00b5ee0 msgr2=0x7fedd00af000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:50.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.157+0000 7fedd7b68640 1 --2- 172.21.15.5:0/2232647707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedd00b5ee0 0x7fedd00af000 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fedd00b0660 tx=0x7fedcc01c3c0 comp rx=0 tx=0).stop 2024-03-18T15:33:50.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.157+0000 7fedd7b68640 1 -- 172.21.15.5:0/2232647707 shutdown_connections 2024-03-18T15:33:50.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.157+0000 7fedd7b68640 1 --2- 172.21.15.5:0/2232647707 >> v2:172.21.15.80:3300/0 conn(0x7fedd0148390 0x7fedd014eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:50.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.157+0000 7fedd7b68640 1 --2- 172.21.15.5:0/2232647707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedd00b5ee0 0x7fedd00af000 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:50.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.157+0000 7fedd7b68640 1 -- 172.21.15.5:0/2232647707 >> 172.21.15.5:0/2232647707 conn(0x7fedd00b9070 msgr2=0x7fedd00b5570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:50.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.157+0000 7fedd7b68640 1 -- 172.21.15.5:0/2232647707 shutdown_connections 2024-03-18T15:33:50.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.157+0000 7fedd7b68640 1 -- 172.21.15.5:0/2232647707 wait complete. 2024-03-18T15:33:50.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.157+0000 7fedd7b68640 1 Processor -- start 2024-03-18T15:33:50.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.157+0000 7fedd7b68640 1 -- start start 2024-03-18T15:33:50.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.157+0000 7fedd7b68640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fedd00b5ee0 0x7fedd006f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:50.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.157+0000 7fedd7b68640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedd0148390 0x7fedd01e36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:50.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.157+0000 7fedd7b68640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedd01e40c0 0x7fedd01419d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:50.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.157+0000 7fedd7b68640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fedd00af6c0 con 0x7fedd01e40c0 2024-03-18T15:33:50.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.157+0000 7fedd7b68640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fedd00af540 con 0x7fedd0148390 2024-03-18T15:33:50.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.157+0000 7fedd7b68640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fedd00b3c80 con 0x7fedd00b5ee0 2024-03-18T15:33:50.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.157+0000 7fedd6105640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedd01e40c0 0x7fedd01419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:50.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.157+0000 7fedd6105640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedd01e40c0 0x7fedd01419d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38654/0 (socket says 172.21.15.5:38654) 2024-03-18T15:33:50.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.157+0000 7fedd6105640 1 -- 172.21.15.5:0/3664680610 learned_addr learned my addr 172.21.15.5:0/3664680610 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:50.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.157+0000 7fedd5103640 1 --2- 172.21.15.5:0/3664680610 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedd0148390 0x7fedd01e36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:50.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.157+0000 7fedd5904640 1 --2- 172.21.15.5:0/3664680610 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fedd00b5ee0 0x7fedd006f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:50.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.157+0000 7fedd6105640 1 -- 172.21.15.5:0/3664680610 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fedd00b5ee0 msgr2=0x7fedd006f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:50.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.157+0000 7fedd6105640 1 --2- 172.21.15.5:0/3664680610 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fedd00b5ee0 0x7fedd006f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:50.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.157+0000 7fedd6105640 1 -- 172.21.15.5:0/3664680610 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedd0148390 msgr2=0x7fedd01e36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:50.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.157+0000 7fedd6105640 1 --2- 172.21.15.5:0/3664680610 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedd0148390 0x7fedd01e36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:50.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.157+0000 7fedd6105640 1 -- 172.21.15.5:0/3664680610 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedd00721e0 con 0x7fedd01e40c0 2024-03-18T15:33:50.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.157+0000 7fedd6105640 1 --2- 172.21.15.5:0/3664680610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedd01e40c0 0x7fedd01419d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fedc800efc0 tx=0x7fedc800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:50.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.157+0000 7fedc27fc640 1 -- 172.21.15.5:0/3664680610 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc8018020 con 0x7fedd01e40c0 2024-03-18T15:33:50.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.157+0000 7fedc27fc640 1 -- 172.21.15.5:0/3664680610 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedc8014480 con 0x7fedd01e40c0 2024-03-18T15:33:50.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.157+0000 7fedd7b68640 1 -- 172.21.15.5:0/3664680610 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedd0071190 con 0x7fedd01e40c0 2024-03-18T15:33:50.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.157+0000 7fedc27fc640 1 -- 172.21.15.5:0/3664680610 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedc8014870 con 0x7fedd01e40c0 2024-03-18T15:33:50.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.157+0000 7fedd7b68640 1 -- 172.21.15.5:0/3664680610 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedd0071f00 con 0x7fedd01e40c0 2024-03-18T15:33:50.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.157+0000 7fedd7b68640 1 -- 172.21.15.5:0/3664680610 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedd0068360 con 0x7fedd01e40c0 2024-03-18T15:33:50.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.161+0000 7fedc27fc640 1 -- 172.21.15.5:0/3664680610 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fedc801b020 con 0x7fedd01e40c0 2024-03-18T15:33:50.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.161+0000 7fedc27fc640 1 --2- 172.21.15.5:0/3664680610 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fedb4067660 0x7fedb4069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:50.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.161+0000 7fedd5904640 1 --2- 172.21.15.5:0/3664680610 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fedb4067660 0x7fedb4069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:50.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.161+0000 7fedc27fc640 1 -- 172.21.15.5:0/3664680610 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fedc8086150 con 0x7fedd01e40c0 2024-03-18T15:33:50.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.161+0000 7fedd5904640 1 --2- 172.21.15.5:0/3664680610 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fedb4067660 0x7fedb4069b20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fedcc0ad000 tx=0x7fedcc0a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:50.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.165+0000 7fedc27fc640 1 -- 172.21.15.5:0/3664680610 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fedc8022070 con 0x7fedd01e40c0 2024-03-18T15:33:50.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.353+0000 7fedd7b68640 1 -- 172.21.15.5:0/3664680610 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fedd00b4d00 con 0x7fedd01e40c0 2024-03-18T15:33:50.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.353+0000 7fedc27fc640 1 -- 172.21.15.5:0/3664680610 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fedd00721e0 con 0x7fedd01e40c0 2024-03-18T15:33:50.355 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:33:50.355 INFO:teuthology.orchestra.run.smithi005.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-18T15:33:50.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.353+0000 7fedd7b68640 1 -- 172.21.15.5:0/3664680610 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fedb4067660 msgr2=0x7fedb4069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:50.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.353+0000 7fedd7b68640 1 --2- 172.21.15.5:0/3664680610 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fedb4067660 0x7fedb4069b20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fedcc0ad000 tx=0x7fedcc0a6000 comp rx=0 tx=0).stop 2024-03-18T15:33:50.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.353+0000 7fedd7b68640 1 -- 172.21.15.5:0/3664680610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedd01e40c0 msgr2=0x7fedd01419d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:50.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.353+0000 7fedd7b68640 1 --2- 172.21.15.5:0/3664680610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedd01e40c0 0x7fedd01419d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fedc800efc0 tx=0x7fedc800c490 comp rx=0 tx=0).stop 2024-03-18T15:33:50.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.353+0000 7fedd7b68640 1 -- 172.21.15.5:0/3664680610 shutdown_connections 2024-03-18T15:33:50.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.353+0000 7fedd7b68640 1 --2- 172.21.15.5:0/3664680610 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fedb4067660 0x7fedb4069b20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:50.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.353+0000 7fedd7b68640 1 --2- 172.21.15.5:0/3664680610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fedd01e40c0 0x7fedd01419d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:50.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.353+0000 7fedd7b68640 1 --2- 172.21.15.5:0/3664680610 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fedd0148390 0x7fedd01e36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:50.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.353+0000 7fedd7b68640 1 --2- 172.21.15.5:0/3664680610 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fedd00b5ee0 0x7fedd006f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:50.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.353+0000 7fedd7b68640 1 -- 172.21.15.5:0/3664680610 >> 172.21.15.5:0/3664680610 conn(0x7fedd00b9070 msgr2=0x7fedd00b7350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:50.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.357+0000 7fedd7b68640 1 -- 172.21.15.5:0/3664680610 shutdown_connections 2024-03-18T15:33:50.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.357+0000 7fedd7b68640 1 -- 172.21.15.5:0/3664680610 wait complete. 2024-03-18T15:33:50.373 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-03-18T15:33:50.373 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:33:50.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.481+0000 7fa67e73a640 1 Processor -- start 2024-03-18T15:33:50.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.481+0000 7fa67e73a640 1 -- start start 2024-03-18T15:33:50.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.485+0000 7fa67e73a640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa67806d2d0 0x7fa6780b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:50.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.485+0000 7fa67e73a640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa67813f260 0x7fa67813f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:50.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.485+0000 7fa67e73a640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6780b0ea0 con 0x7fa6781453a0 2024-03-18T15:33:50.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.485+0000 7fa67e73a640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fa6780728c0 con 0x7fa67806d2d0 2024-03-18T15:33:50.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.485+0000 7fa67e73a640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fa67806d6b0 con 0x7fa67813f260 2024-03-18T15:33:50.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.485+0000 7fa677fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa67806d2d0 0x7fa6780b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:50.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.485+0000 7fa67ccd7640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa6781453a0 0x7fa678145810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46840/0 (socket says 172.21.15.5:46840) 2024-03-18T15:33:50.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.485+0000 7fa67ccd7640 1 -- 172.21.15.5:0/3383232157 learned_addr learned my addr 172.21.15.5:0/3383232157 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:33:50.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.485+0000 7fa677fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa67806d2d0 0x7fa6780b23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42836/0 (socket says 172.21.15.5:42836) 2024-03-18T15:33:50.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.485+0000 7fa6777fe640 1 --2- 172.21.15.5:0/3383232157 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa67813f260 0x7fa67813f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:50.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.485+0000 7fa677fff640 1 -- 172.21.15.5:0/3383232157 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa67813f260 msgr2=0x7fa67813f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:50.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.485+0000 7fa676ffd640 1 -- 172.21.15.5:0/3383232157 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 595370840 0 0) 0x7fa6780b0ea0 con 0x7fa6781453a0 2024-03-18T15:33:50.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.485+0000 7fa677fff640 1 --2- 172.21.15.5:0/3383232157 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa67813f260 0x7fa67813f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:50.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.485+0000 7fa677fff640 1 -- 172.21.15.5:0/3383232157 >> v1:172.21.15.5:6789/0 conn(0x7fa6781453a0 legacy=0x7fa678145810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:50.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.485+0000 7fa677fff640 1 -- 172.21.15.5:0/3383232157 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6780b0bc0 con 0x7fa67806d2d0 2024-03-18T15:33:50.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.485+0000 7fa6777fe640 1 --2- 172.21.15.5:0/3383232157 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa67813f260 0x7fa67813f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:33:50.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.485+0000 7fa677fff640 1 --2- 172.21.15.5:0/3383232157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa67806d2d0 0x7fa6780b23a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa6680097c0 tx=0x7fa66801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31d5c489cb5e244f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:50.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.485+0000 7fa676ffd640 1 -- 172.21.15.5:0/3383232157 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6680b0070 con 0x7fa67806d2d0 2024-03-18T15:33:50.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.485+0000 7fa676ffd640 1 -- 172.21.15.5:0/3383232157 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6680ab600 con 0x7fa67806d2d0 2024-03-18T15:33:50.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.485+0000 7fa676ffd640 1 -- 172.21.15.5:0/3383232157 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6680ab9d0 con 0x7fa67806d2d0 2024-03-18T15:33:50.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.485+0000 7fa67e73a640 1 -- 172.21.15.5:0/3383232157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa67806d2d0 msgr2=0x7fa6780b23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:50.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.485+0000 7fa67e73a640 1 --2- 172.21.15.5:0/3383232157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa67806d2d0 0x7fa6780b23a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa6680097c0 tx=0x7fa66801c410 comp rx=0 tx=0).stop 2024-03-18T15:33:50.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.485+0000 7fa67e73a640 1 -- 172.21.15.5:0/3383232157 shutdown_connections 2024-03-18T15:33:50.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.485+0000 7fa67e73a640 1 --2- 172.21.15.5:0/3383232157 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa67813f260 0x7fa67813f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:50.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.485+0000 7fa67e73a640 1 --2- 172.21.15.5:0/3383232157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa67806d2d0 0x7fa6780b23a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:50.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.485+0000 7fa67e73a640 1 -- 172.21.15.5:0/3383232157 >> 172.21.15.5:0/3383232157 conn(0x7fa6780b5670 msgr2=0x7fa6780b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:50.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.485+0000 7fa67e73a640 1 -- 172.21.15.5:0/3383232157 shutdown_connections 2024-03-18T15:33:50.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.485+0000 7fa67e73a640 1 -- 172.21.15.5:0/3383232157 wait complete. 2024-03-18T15:33:50.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.489+0000 7fa67e73a640 1 Processor -- start 2024-03-18T15:33:50.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.489+0000 7fa67e73a640 1 -- start start 2024-03-18T15:33:50.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.489+0000 7fa67e73a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa67806d2d0 0x7fa6780b7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:50.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.489+0000 7fa67e73a640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa67813f260 0x7fa6781e7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:50.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.489+0000 7fa67e73a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6781453a0 0x7fa678135f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:50.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.489+0000 7fa67e73a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa678072ed0 con 0x7fa67806d2d0 2024-03-18T15:33:50.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.489+0000 7fa67e73a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa678072d50 con 0x7fa6781453a0 2024-03-18T15:33:50.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.489+0000 7fa67e73a640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fa6780b5140 con 0x7fa67813f260 2024-03-18T15:33:50.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.489+0000 7fa677fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa67806d2d0 0x7fa6780b7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:50.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.489+0000 7fa677fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa67806d2d0 0x7fa6780b7c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42850/0 (socket says 172.21.15.5:42850) 2024-03-18T15:33:50.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.489+0000 7fa677fff640 1 -- 172.21.15.5:0/1586826567 learned_addr learned my addr 172.21.15.5:0/1586826567 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:50.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.489+0000 7fa67ccd7640 1 --2- 172.21.15.5:0/1586826567 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6781453a0 0x7fa678135f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:50.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.489+0000 7fa6777fe640 1 --2- 172.21.15.5:0/1586826567 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa67813f260 0x7fa6781e7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:50.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.489+0000 7fa677fff640 1 -- 172.21.15.5:0/1586826567 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa67813f260 msgr2=0x7fa6781e7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:50.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.489+0000 7fa677fff640 1 --2- 172.21.15.5:0/1586826567 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa67813f260 0x7fa6781e7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:50.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.489+0000 7fa677fff640 1 -- 172.21.15.5:0/1586826567 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6781453a0 msgr2=0x7fa678135f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:50.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.489+0000 7fa677fff640 1 --2- 172.21.15.5:0/1586826567 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6781453a0 0x7fa678135f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:50.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.489+0000 7fa677fff640 1 -- 172.21.15.5:0/1586826567 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6780af150 con 0x7fa67806d2d0 2024-03-18T15:33:50.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.489+0000 7fa677fff640 1 --2- 172.21.15.5:0/1586826567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa67806d2d0 0x7fa6780b7c20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa6680097c0 tx=0x7fa6680191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:50.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.489+0000 7fa674ff9640 1 -- 172.21.15.5:0/1586826567 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6680b8020 con 0x7fa67806d2d0 2024-03-18T15:33:50.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.489+0000 7fa674ff9640 1 -- 172.21.15.5:0/1586826567 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6680b4630 con 0x7fa67806d2d0 2024-03-18T15:33:50.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.489+0000 7fa67e73a640 1 -- 172.21.15.5:0/1586826567 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa678142ff0 con 0x7fa67806d2d0 2024-03-18T15:33:50.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.489+0000 7fa674ff9640 1 -- 172.21.15.5:0/1586826567 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6680b0040 con 0x7fa67806d2d0 2024-03-18T15:33:50.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.489+0000 7fa67e73a640 1 -- 172.21.15.5:0/1586826567 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6780b8550 con 0x7fa67806d2d0 2024-03-18T15:33:50.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.489+0000 7fa67e73a640 1 -- 172.21.15.5:0/1586826567 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa67804f550 con 0x7fa67806d2d0 2024-03-18T15:33:50.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.489+0000 7fa674ff9640 1 -- 172.21.15.5:0/1586826567 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fa6680bb020 con 0x7fa67806d2d0 2024-03-18T15:33:50.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.493+0000 7fa674ff9640 1 --2- 172.21.15.5:0/1586826567 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa664067660 0x7fa664069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:50.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.493+0000 7fa674ff9640 1 -- 172.21.15.5:0/1586826567 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa66811f390 con 0x7fa67806d2d0 2024-03-18T15:33:50.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.493+0000 7fa6777fe640 1 --2- 172.21.15.5:0/1586826567 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa664067660 0x7fa664069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:50.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.493+0000 7fa6777fe640 1 --2- 172.21.15.5:0/1586826567 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa664067660 0x7fa664069b20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa66c006fd0 tx=0x7fa66c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:50.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.497+0000 7fa674ff9640 1 -- 172.21.15.5:0/1586826567 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fa6680c2030 con 0x7fa67806d2d0 2024-03-18T15:33:50.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.649+0000 7fa67e73a640 1 -- 172.21.15.5:0/1586826567 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa6780b2de0 con 0x7fa67806d2d0 2024-03-18T15:33:50.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.653+0000 7fa674ff9640 1 -- 172.21.15.5:0/1586826567 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16449 (secure 0 0 0) 0x7fa668016090 con 0x7fa67806d2d0 2024-03-18T15:33:50.654 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:33:50.654 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":21,"fsid":"ef54fe03-5aa7-4eb2-9b1e-66c9cc345b94","created":"2024-03-18T15:33:18.086614+0000","modified":"2024-03-18T15:33:43.984050+0000","last_up_change":"2024-03-18T15:33:29.229911+0000","last_in_change":"2024-03-18T15:33:20.323840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:28.963186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T15:33:40.533646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T15:33:41.323519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df1c754b-af59-4d16-8919-fbb6386b3c48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6804","nonce":976264754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6808","nonce":976264754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6815","nonce":976264754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6811","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6813","nonce":976264754}]},"public_addr":"172.21.15.5:6804/976264754","cluster_addr":"172.21.15.5:6808/976264754","heartbeat_back_addr":"172.21.15.5:6815/976264754","heartbeat_front_addr":"172.21.15.5:6813/976264754","state":["exists","up"]},{"osd":1,"uuid":"b9a08b5e-8af4-4144-bdba-d69ed1ac627a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6801","nonce":3901761126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6803","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6805","nonce":3901761126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6812","nonce":3901761126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6807","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6809","nonce":3901761126}]},"public_addr":"172.21.15.5:6801/3901761126","cluster_addr":"172.21.15.5:6805/3901761126","heartbeat_back_addr":"172.21.15.5:6812/3901761126","heartbeat_front_addr":"172.21.15.5:6809/3901761126","state":["exists","up"]},{"osd":2,"uuid":"c3e8bb76-e382-439d-b380-62ef92f1255a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6817","nonce":502968448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6819","nonce":502968448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6823","nonce":502968448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6821","nonce":502968448}]},"public_addr":"172.21.15.5:6817/502968448","cluster_addr":"172.21.15.5:6819/502968448","heartbeat_back_addr":"172.21.15.5:6823/502968448","heartbeat_front_addr":"172.21.15.5:6821/502968448","state":["exists","up"]},{"osd":3,"uuid":"e8c94625-f2cd-4eba-b53d-2e7356e1c02a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6825","nonce":2425932735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6827","nonce":2425932735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6831","nonce":2425932735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6829","nonce":2425932735}]},"public_addr":"172.21.15.5:6825/2425932735","cluster_addr":"172.21.15.5:6827/2425932735","heartbeat_back_addr":"172.21.15.5:6831/2425932735","heartbeat_front_addr":"172.21.15.5:6829/2425932735","state":["exists","up"]},{"osd":4,"uuid":"0099dd52-6265-4d32-82e4-3ef16e73add0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6817","nonce":976514285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6819","nonce":976514285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6823","nonce":976514285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6821","nonce":976514285}]},"public_addr":"172.21.15.80:6817/976514285","cluster_addr":"172.21.15.80:6819/976514285","heartbeat_back_addr":"172.21.15.80:6823/976514285","heartbeat_front_addr":"172.21.15.80:6821/976514285","state":["exists","up"]},{"osd":5,"uuid":"ed10c84f-8c39-41cc-9b64-7c482a94a48e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6825","nonce":627153447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6827","nonce":627153447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6831","nonce":627153447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6829","nonce":627153447}]},"public_addr":"172.21.15.80:6825/627153447","cluster_addr":"172.21.15.80:6827/627153447","heartbeat_back_addr":"172.21.15.80:6831/627153447","heartbeat_front_addr":"172.21.15.80:6829/627153447","state":["exists","up"]},{"osd":6,"uuid":"c7a3538a-7048-4fba-8781-b440f03c25db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6803","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6805","nonce":488342527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6807","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6809","nonce":488342527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6815","nonce":488342527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6811","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6813","nonce":488342527}]},"public_addr":"172.21.15.80:6805/488342527","cluster_addr":"172.21.15.80:6809/488342527","heartbeat_back_addr":"172.21.15.80:6815/488342527","heartbeat_front_addr":"172.21.15.80:6813/488342527","state":["exists","up"]},{"osd":7,"uuid":"f9418104-3a97-4110-bc3a-1f8151d0f757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6801","nonce":982831666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6804","nonce":982831666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6812","nonce":982831666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6808","nonce":982831666}]},"public_addr":"172.21.15.80:6801/982831666","cluster_addr":"172.21.15.80:6804/982831666","heartbeat_back_addr":"172.21.15.80:6812/982831666","heartbeat_front_addr":"172.21.15.80:6808/982831666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.970828+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.858116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.987556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.864603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.756293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.591502+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:33:50.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.653+0000 7fa67e73a640 1 -- 172.21.15.5:0/1586826567 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa664067660 msgr2=0x7fa664069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:50.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.653+0000 7fa67e73a640 1 --2- 172.21.15.5:0/1586826567 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa664067660 0x7fa664069b20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa66c006fd0 tx=0x7fa66c008040 comp rx=0 tx=0).stop 2024-03-18T15:33:50.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.653+0000 7fa67e73a640 1 -- 172.21.15.5:0/1586826567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa67806d2d0 msgr2=0x7fa6780b7c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:50.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.653+0000 7fa67e73a640 1 --2- 172.21.15.5:0/1586826567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa67806d2d0 0x7fa6780b7c20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa6680097c0 tx=0x7fa6680191b0 comp rx=0 tx=0).stop 2024-03-18T15:33:50.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.653+0000 7fa67e73a640 1 -- 172.21.15.5:0/1586826567 shutdown_connections 2024-03-18T15:33:50.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.653+0000 7fa67e73a640 1 --2- 172.21.15.5:0/1586826567 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa664067660 0x7fa664069b20 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:50.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.653+0000 7fa67e73a640 1 --2- 172.21.15.5:0/1586826567 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6781453a0 0x7fa678135f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:50.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.653+0000 7fa67e73a640 1 --2- 172.21.15.5:0/1586826567 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa67813f260 0x7fa6781e7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:50.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.653+0000 7fa67e73a640 1 --2- 172.21.15.5:0/1586826567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa67806d2d0 0x7fa6780b7c20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:50.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.653+0000 7fa67e73a640 1 -- 172.21.15.5:0/1586826567 >> 172.21.15.5:0/1586826567 conn(0x7fa6780b5670 msgr2=0x7fa6780b5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:50.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.653+0000 7fa67e73a640 1 -- 172.21.15.5:0/1586826567 shutdown_connections 2024-03-18T15:33:50.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.653+0000 7fa67e73a640 1 -- 172.21.15.5:0/1586826567 wait complete. 2024-03-18T15:33:50.671 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T15:33:28.963186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-18T15:33:40.533646+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-18T15:33:41.323519+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T15:33:50.672 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T15:33:50.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.805+0000 7f6cd29e9640 1 Processor -- start 2024-03-18T15:33:50.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.805+0000 7f6cd29e9640 1 -- start start 2024-03-18T15:33:50.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.805+0000 7f6cd29e9640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6ccc14b030 0x7f6ccc140ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:50.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.805+0000 7f6cd29e9640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6ccc141430 0x7f6ccc151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:50.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.805+0000 7f6cd29e9640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ccc0b9d50 con 0x7f6ccc0b5710 2024-03-18T15:33:50.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.805+0000 7f6cd29e9640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f6ccc0b4320 con 0x7f6ccc141430 2024-03-18T15:33:50.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.805+0000 7f6cd29e9640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f6ccc0b4570 con 0x7f6ccc14b030 2024-03-18T15:33:50.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.805+0000 7f6ccbfff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f6ccc0b5710 0x7f6ccc14a920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46854/0 (socket says 172.21.15.5:46854) 2024-03-18T15:33:50.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.805+0000 7f6ccbfff640 1 -- 172.21.15.5:0/74362644 learned_addr learned my addr 172.21.15.5:0/74362644 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:33:50.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.805+0000 7f6cd0f86640 1 --2- 172.21.15.5:0/74362644 >> v2:172.21.15.5:3300/0 conn(0x7f6ccc141430 0x7f6ccc151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:50.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.805+0000 7f6cc37fe640 1 --2- 172.21.15.5:0/74362644 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6ccc14b030 0x7f6ccc140ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:50.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.805+0000 7f6cd0f86640 1 -- 172.21.15.5:0/74362644 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6ccc14b030 msgr2=0x7f6ccc140ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:50.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.805+0000 7f6cd0f86640 1 --2- 172.21.15.5:0/74362644 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6ccc14b030 0x7f6ccc140ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:50.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.805+0000 7f6cd0f86640 1 -- 172.21.15.5:0/74362644 >> v1:172.21.15.5:6789/0 conn(0x7f6ccc0b5710 legacy=0x7f6ccc14a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:50.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.805+0000 7f6cd0f86640 1 -- 172.21.15.5:0/74362644 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ccc0b4c30 con 0x7f6ccc141430 2024-03-18T15:33:50.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.805+0000 7f6cd0f86640 1 --2- 172.21.15.5:0/74362644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ccc141430 0x7f6ccc151cb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4009dc0 tx=0x7f6cc401c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f51a24333a181bb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:50.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.809+0000 7f6ccb7fe640 1 -- 172.21.15.5:0/74362644 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc40a6700 con 0x7f6ccc141430 2024-03-18T15:33:50.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.809+0000 7f6ccb7fe640 1 -- 172.21.15.5:0/74362644 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cc40a68a0 con 0x7f6ccc141430 2024-03-18T15:33:50.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.809+0000 7f6ccb7fe640 1 -- 172.21.15.5:0/74362644 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc40a6c70 con 0x7f6ccc141430 2024-03-18T15:33:50.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.809+0000 7f6cd29e9640 1 -- 172.21.15.5:0/74362644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ccc141430 msgr2=0x7f6ccc151cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:50.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.809+0000 7f6cd29e9640 1 --2- 172.21.15.5:0/74362644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ccc141430 0x7f6ccc151cb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4009dc0 tx=0x7f6cc401c830 comp rx=0 tx=0).stop 2024-03-18T15:33:50.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.809+0000 7f6cd29e9640 1 -- 172.21.15.5:0/74362644 shutdown_connections 2024-03-18T15:33:50.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.809+0000 7f6cd29e9640 1 --2- 172.21.15.5:0/74362644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ccc141430 0x7f6ccc151cb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:50.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.809+0000 7f6cd29e9640 1 --2- 172.21.15.5:0/74362644 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6ccc14b030 0x7f6ccc140ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:50.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.809+0000 7f6cd29e9640 1 -- 172.21.15.5:0/74362644 >> 172.21.15.5:0/74362644 conn(0x7f6ccc0673c0 msgr2=0x7f6ccc0677d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:50.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.809+0000 7f6cd29e9640 1 -- 172.21.15.5:0/74362644 shutdown_connections 2024-03-18T15:33:50.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.809+0000 7f6cd29e9640 1 -- 172.21.15.5:0/74362644 wait complete. 2024-03-18T15:33:50.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.809+0000 7f6cd29e9640 1 Processor -- start 2024-03-18T15:33:50.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.809+0000 7f6cd29e9640 1 -- start start 2024-03-18T15:33:50.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.809+0000 7f6cd29e9640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6ccc0b5710 0x7f6ccc13a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:50.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.809+0000 7f6cd29e9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ccc141430 0x7f6ccc131ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:50.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.809+0000 7f6cd29e9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ccc14b030 0x7f6ccc1e0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:50.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.809+0000 7f6cd29e9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6ccc0b5260 con 0x7f6ccc141430 2024-03-18T15:33:50.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.809+0000 7f6cd29e9640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f6ccc0b50e0 con 0x7f6ccc14b030 2024-03-18T15:33:50.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.809+0000 7f6cd29e9640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f6ccc154a10 con 0x7f6ccc0b5710 2024-03-18T15:33:50.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.809+0000 7f6cc37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ccc141430 0x7f6ccc131ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:50.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.809+0000 7f6cc37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ccc141430 0x7f6ccc131ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42860/0 (socket says 172.21.15.5:42860) 2024-03-18T15:33:50.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.809+0000 7f6cc37fe640 1 -- 172.21.15.5:0/97200740 learned_addr learned my addr 172.21.15.5:0/97200740 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:50.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.809+0000 7f6ccbfff640 1 --2- 172.21.15.5:0/97200740 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6ccc0b5710 0x7f6ccc13a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:50.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.809+0000 7f6cd0f86640 1 --2- 172.21.15.5:0/97200740 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ccc14b030 0x7f6ccc1e0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:50.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.809+0000 7f6cc37fe640 1 -- 172.21.15.5:0/97200740 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6ccc0b5710 msgr2=0x7f6ccc13a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:50.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.809+0000 7f6cc37fe640 1 --2- 172.21.15.5:0/97200740 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6ccc0b5710 0x7f6ccc13a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:50.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.809+0000 7f6cc37fe640 1 -- 172.21.15.5:0/97200740 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ccc14b030 msgr2=0x7f6ccc1e0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:50.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.809+0000 7f6cc37fe640 1 --2- 172.21.15.5:0/97200740 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ccc14b030 0x7f6ccc1e0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:50.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.809+0000 7f6cc37fe640 1 -- 172.21.15.5:0/97200740 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ccc13af60 con 0x7f6ccc141430 2024-03-18T15:33:50.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.809+0000 7f6cd0f86640 1 --2- 172.21.15.5:0/97200740 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ccc14b030 0x7f6ccc1e0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:33:50.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.809+0000 7f6ccbfff640 1 --2- 172.21.15.5:0/97200740 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6ccc0b5710 0x7f6ccc13a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:33:50.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.809+0000 7f6cc37fe640 1 --2- 172.21.15.5:0/97200740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ccc141430 0x7f6ccc131ce0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6cb400d7d0 tx=0x7f6cb400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:50.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.809+0000 7f6cc97fa640 1 -- 172.21.15.5:0/97200740 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb400b660 con 0x7f6ccc141430 2024-03-18T15:33:50.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.809+0000 7f6cc97fa640 1 -- 172.21.15.5:0/97200740 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cb4019070 con 0x7f6ccc141430 2024-03-18T15:33:50.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.809+0000 7f6cd29e9640 1 -- 172.21.15.5:0/97200740 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ccc1dfe40 con 0x7f6ccc141430 2024-03-18T15:33:50.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.809+0000 7f6cc97fa640 1 -- 172.21.15.5:0/97200740 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb400ba20 con 0x7f6ccc141430 2024-03-18T15:33:50.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.809+0000 7f6cd29e9640 1 -- 172.21.15.5:0/97200740 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ccc1e0100 con 0x7f6ccc141430 2024-03-18T15:33:50.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.813+0000 7f6cd29e9640 1 -- 172.21.15.5:0/97200740 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ccc04f550 con 0x7f6ccc141430 2024-03-18T15:33:50.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.813+0000 7f6cc97fa640 1 -- 172.21.15.5:0/97200740 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f6cb400bbc0 con 0x7f6ccc141430 2024-03-18T15:33:50.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.813+0000 7f6cc97fa640 1 --2- 172.21.15.5:0/97200740 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6cac067690 0x7f6cac069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:50.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.813+0000 7f6cc97fa640 1 -- 172.21.15.5:0/97200740 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f6cb4085b80 con 0x7f6ccc141430 2024-03-18T15:33:50.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.813+0000 7f6ccbfff640 1 --2- 172.21.15.5:0/97200740 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6cac067690 0x7f6cac069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:50.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.813+0000 7f6ccbfff640 1 --2- 172.21.15.5:0/97200740 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6cac067690 0x7f6cac069b50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc000f70 tx=0x7f6cbc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:50.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.817+0000 7f6cc97fa640 1 -- 172.21.15.5:0/97200740 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f6cb4056890 con 0x7f6ccc141430 2024-03-18T15:33:50.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.977+0000 7f6cd29e9640 1 -- 172.21.15.5:0/97200740 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6ccc133ce0 con 0x7f6ccc141430 2024-03-18T15:33:50.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.977+0000 7f6cc97fa640 1 -- 172.21.15.5:0/97200740 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f6cb4056190 con 0x7f6ccc141430 2024-03-18T15:33:50.981 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-03-18T15:33:50.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.981+0000 7f6cd29e9640 1 -- 172.21.15.5:0/97200740 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6cac067690 msgr2=0x7f6cac069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:50.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.981+0000 7f6cd29e9640 1 --2- 172.21.15.5:0/97200740 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6cac067690 0x7f6cac069b50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc000f70 tx=0x7f6cbc009040 comp rx=0 tx=0).stop 2024-03-18T15:33:50.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.981+0000 7f6cd29e9640 1 -- 172.21.15.5:0/97200740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ccc141430 msgr2=0x7f6ccc131ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:50.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.981+0000 7f6cd29e9640 1 --2- 172.21.15.5:0/97200740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ccc141430 0x7f6ccc131ce0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6cb400d7d0 tx=0x7f6cb400dc90 comp rx=0 tx=0).stop 2024-03-18T15:33:50.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.981+0000 7f6cd29e9640 1 -- 172.21.15.5:0/97200740 shutdown_connections 2024-03-18T15:33:50.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.981+0000 7f6cd29e9640 1 --2- 172.21.15.5:0/97200740 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6cac067690 0x7f6cac069b50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:50.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.981+0000 7f6cd29e9640 1 --2- 172.21.15.5:0/97200740 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ccc14b030 0x7f6ccc1e0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:50.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.981+0000 7f6cd29e9640 1 --2- 172.21.15.5:0/97200740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ccc141430 0x7f6ccc131ce0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:50.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.981+0000 7f6cd29e9640 1 --2- 172.21.15.5:0/97200740 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6ccc0b5710 0x7f6ccc13a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:50.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.981+0000 7f6cd29e9640 1 -- 172.21.15.5:0/97200740 >> 172.21.15.5:0/97200740 conn(0x7f6ccc0673c0 msgr2=0x7f6ccc13cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:50.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.981+0000 7f6cd29e9640 1 -- 172.21.15.5:0/97200740 shutdown_connections 2024-03-18T15:33:50.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:50.981+0000 7f6cd29e9640 1 -- 172.21.15.5:0/97200740 wait complete. 2024-03-18T15:33:50.999 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-18T15:33:51.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.125+0000 7f716a772640 1 Processor -- start 2024-03-18T15:33:51.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.125+0000 7f716a772640 1 -- start start 2024-03-18T15:33:51.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.125+0000 7f716a772640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f71640ba040 0x7f71640b5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:51.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.125+0000 7f716a772640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f7164141700 0x7f7164151dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:51.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.129+0000 7f716a772640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71640ba420 con 0x7f716414aa40 2024-03-18T15:33:51.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.129+0000 7f716a772640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f71640b4fa0 con 0x7f71640ba040 2024-03-18T15:33:51.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.129+0000 7f716a772640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f71640b44c0 con 0x7f7164141700 2024-03-18T15:33:51.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.129+0000 7f71637fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f716414aa40 0x7f7164140ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46864/0 (socket says 172.21.15.5:46864) 2024-03-18T15:33:51.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.129+0000 7f71637fe640 1 -- 172.21.15.5:0/3306359226 learned_addr learned my addr 172.21.15.5:0/3306359226 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:33:51.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.129+0000 7f7163fff640 1 --2- 172.21.15.5:0/3306359226 >> v2:172.21.15.5:3300/0 conn(0x7f71640ba040 0x7f71640b5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:51.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.129+0000 7f7168d0f640 1 --2- 172.21.15.5:0/3306359226 >> v2:172.21.15.80:3300/0 conn(0x7f7164141700 0x7f7164151dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:51.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.129+0000 7f7162ffd640 1 -- 172.21.15.5:0/3306359226 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2278093551 0 0) 0x7f71640ba420 con 0x7f716414aa40 2024-03-18T15:33:51.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.129+0000 7f7162ffd640 1 -- 172.21.15.5:0/3306359226 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f714c003530 con 0x7f716414aa40 2024-03-18T15:33:51.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.129+0000 7f7163fff640 1 -- 172.21.15.5:0/3306359226 >> v2:172.21.15.80:3300/0 conn(0x7f7164141700 msgr2=0x7f7164151dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:51.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.129+0000 7f7163fff640 1 --2- 172.21.15.5:0/3306359226 >> v2:172.21.15.80:3300/0 conn(0x7f7164141700 0x7f7164151dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:51.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.129+0000 7f7163fff640 1 -- 172.21.15.5:0/3306359226 >> v1:172.21.15.5:6789/0 conn(0x7f716414aa40 legacy=0x7f7164140ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:51.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.129+0000 7f7163fff640 1 -- 172.21.15.5:0/3306359226 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71640b47a0 con 0x7f71640ba040 2024-03-18T15:33:51.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.129+0000 7f7163fff640 1 --2- 172.21.15.5:0/3306359226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71640ba040 0x7f71640b5800 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f71640b8d70 tx=0x7f715401c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbcd2d64478d60f9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:51.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.129+0000 7f7162ffd640 1 -- 172.21.15.5:0/3306359226 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71540b0070 con 0x7f71640ba040 2024-03-18T15:33:51.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.129+0000 7f7162ffd640 1 -- 172.21.15.5:0/3306359226 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71540ab600 con 0x7f71640ba040 2024-03-18T15:33:51.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.129+0000 7f7162ffd640 1 -- 172.21.15.5:0/3306359226 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71540ab9d0 con 0x7f71640ba040 2024-03-18T15:33:51.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.129+0000 7f716a772640 1 -- 172.21.15.5:0/3306359226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71640ba040 msgr2=0x7f71640b5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:51.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.129+0000 7f716a772640 1 --2- 172.21.15.5:0/3306359226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71640ba040 0x7f71640b5800 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f71640b8d70 tx=0x7f715401c360 comp rx=0 tx=0).stop 2024-03-18T15:33:51.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.129+0000 7f716a772640 1 -- 172.21.15.5:0/3306359226 shutdown_connections 2024-03-18T15:33:51.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.129+0000 7f716a772640 1 --2- 172.21.15.5:0/3306359226 >> v2:172.21.15.80:3300/0 conn(0x7f7164141700 0x7f7164151dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:51.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.129+0000 7f716a772640 1 --2- 172.21.15.5:0/3306359226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71640ba040 0x7f71640b5800 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:51.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.129+0000 7f716a772640 1 -- 172.21.15.5:0/3306359226 >> 172.21.15.5:0/3306359226 conn(0x7f71640673c0 msgr2=0x7f71640677d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:51.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.129+0000 7f716a772640 1 -- 172.21.15.5:0/3306359226 shutdown_connections 2024-03-18T15:33:51.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.129+0000 7f716a772640 1 -- 172.21.15.5:0/3306359226 wait complete. 2024-03-18T15:33:51.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.133+0000 7f716a772640 1 Processor -- start 2024-03-18T15:33:51.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.133+0000 7f716a772640 1 -- start start 2024-03-18T15:33:51.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.133+0000 7f716a772640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71640ba040 0x7f71641e06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:51.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.133+0000 7f716a772640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7164141700 0x7f7164142170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:51.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.133+0000 7f716a772640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f716414aa40 0x7f7164149f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:51.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.133+0000 7f716a772640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f71640b95b0 con 0x7f716414aa40 2024-03-18T15:33:51.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.133+0000 7f716a772640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f71640b9430 con 0x7f71640ba040 2024-03-18T15:33:51.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.133+0000 7f716a772640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f7164154b30 con 0x7f7164141700 2024-03-18T15:33:51.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.133+0000 7f7168d0f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f716414aa40 0x7f7164149f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:51.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.133+0000 7f7168d0f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f716414aa40 0x7f7164149f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42880/0 (socket says 172.21.15.5:42880) 2024-03-18T15:33:51.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.133+0000 7f7168d0f640 1 -- 172.21.15.5:0/2426562800 learned_addr learned my addr 172.21.15.5:0/2426562800 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:51.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.133+0000 7f71637fe640 1 --2- 172.21.15.5:0/2426562800 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7164141700 0x7f7164142170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:51.136 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.133+0000 7f7163fff640 1 --2- 172.21.15.5:0/2426562800 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71640ba040 0x7f71641e06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:51.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.133+0000 7f7168d0f640 1 -- 172.21.15.5:0/2426562800 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7164141700 msgr2=0x7f7164142170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:51.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.133+0000 7f7168d0f640 1 --2- 172.21.15.5:0/2426562800 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7164141700 0x7f7164142170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:51.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.133+0000 7f7168d0f640 1 -- 172.21.15.5:0/2426562800 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71640ba040 msgr2=0x7f71641e06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:51.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.133+0000 7f7168d0f640 1 --2- 172.21.15.5:0/2426562800 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71640ba040 0x7f71641e06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:51.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.133+0000 7f7168d0f640 1 -- 172.21.15.5:0/2426562800 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7164144300 con 0x7f716414aa40 2024-03-18T15:33:51.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.133+0000 7f71637fe640 1 --2- 172.21.15.5:0/2426562800 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7164141700 0x7f7164142170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:33:51.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.133+0000 7f7163fff640 1 --2- 172.21.15.5:0/2426562800 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71640ba040 0x7f71641e06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:33:51.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.133+0000 7f7168d0f640 1 --2- 172.21.15.5:0/2426562800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f716414aa40 0x7f7164149f60 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f715c00efc0 tx=0x7f715c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:51.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.133+0000 7f7160ff9640 1 -- 172.21.15.5:0/2426562800 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715c018020 con 0x7f716414aa40 2024-03-18T15:33:51.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.133+0000 7f7160ff9640 1 -- 172.21.15.5:0/2426562800 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f715c014480 con 0x7f716414aa40 2024-03-18T15:33:51.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.133+0000 7f716a772640 1 -- 172.21.15.5:0/2426562800 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71641dfbc0 con 0x7f716414aa40 2024-03-18T15:33:51.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.133+0000 7f7160ff9640 1 -- 172.21.15.5:0/2426562800 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715c014870 con 0x7f716414aa40 2024-03-18T15:33:51.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.133+0000 7f716a772640 1 -- 172.21.15.5:0/2426562800 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71641e0180 con 0x7f716414aa40 2024-03-18T15:33:51.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.133+0000 7f716a772640 1 -- 172.21.15.5:0/2426562800 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f716404f550 con 0x7f716414aa40 2024-03-18T15:33:51.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.133+0000 7f7160ff9640 1 -- 172.21.15.5:0/2426562800 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f715c01b020 con 0x7f716414aa40 2024-03-18T15:33:51.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.137+0000 7f7160ff9640 1 --2- 172.21.15.5:0/2426562800 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f7150067660 0x7f7150069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:51.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.137+0000 7f7160ff9640 1 -- 172.21.15.5:0/2426562800 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f715c086430 con 0x7f716414aa40 2024-03-18T15:33:51.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.137+0000 7f7163fff640 1 --2- 172.21.15.5:0/2426562800 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f7150067660 0x7f7150069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:51.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.137+0000 7f7163fff640 1 --2- 172.21.15.5:0/2426562800 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f7150067660 0x7f7150069b20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f71540ad000 tx=0x7f7154008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:51.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.141+0000 7f7160ff9640 1 -- 172.21.15.5:0/2426562800 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f715c022030 con 0x7f716414aa40 2024-03-18T15:33:51.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.293+0000 7f716a772640 1 -- 172.21.15.5:0/2426562800 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f71640ba420 con 0x7f716414aa40 2024-03-18T15:33:51.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.297+0000 7f7160ff9640 1 -- 172.21.15.5:0/2426562800 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f715c010040 con 0x7f716414aa40 2024-03-18T15:33:51.298 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-03-18T15:33:51.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.297+0000 7f716a772640 1 -- 172.21.15.5:0/2426562800 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f7150067660 msgr2=0x7f7150069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:51.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.297+0000 7f716a772640 1 --2- 172.21.15.5:0/2426562800 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f7150067660 0x7f7150069b20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f71540ad000 tx=0x7f7154008040 comp rx=0 tx=0).stop 2024-03-18T15:33:51.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.297+0000 7f716a772640 1 -- 172.21.15.5:0/2426562800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f716414aa40 msgr2=0x7f7164149f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:51.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.297+0000 7f716a772640 1 --2- 172.21.15.5:0/2426562800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f716414aa40 0x7f7164149f60 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f715c00efc0 tx=0x7f715c00c490 comp rx=0 tx=0).stop 2024-03-18T15:33:51.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.297+0000 7f716a772640 1 -- 172.21.15.5:0/2426562800 shutdown_connections 2024-03-18T15:33:51.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.297+0000 7f716a772640 1 --2- 172.21.15.5:0/2426562800 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f7150067660 0x7f7150069b20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:51.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.297+0000 7f716a772640 1 --2- 172.21.15.5:0/2426562800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f716414aa40 0x7f7164149f60 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:51.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.297+0000 7f716a772640 1 --2- 172.21.15.5:0/2426562800 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7164141700 0x7f7164142170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:51.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.297+0000 7f716a772640 1 --2- 172.21.15.5:0/2426562800 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71640ba040 0x7f71641e06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:51.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.297+0000 7f716a772640 1 -- 172.21.15.5:0/2426562800 >> 172.21.15.5:0/2426562800 conn(0x7f71640673c0 msgr2=0x7f71640bad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:51.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.297+0000 7f716a772640 1 -- 172.21.15.5:0/2426562800 shutdown_connections 2024-03-18T15:33:51.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.297+0000 7f716a772640 1 -- 172.21.15.5:0/2426562800 wait complete. 2024-03-18T15:33:51.316 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-18T15:33:51.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.429+0000 7f1f1467d640 1 Processor -- start 2024-03-18T15:33:51.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.429+0000 7f1f1467d640 1 -- start start 2024-03-18T15:33:51.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.429+0000 7f1f1467d640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1f0c06b4e0 0x7f1f0c0702b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:51.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.429+0000 7f1f1467d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1f0c147bd0 0x7f1f0c1417e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:51.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.429+0000 7f1f1467d640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f0c06b8c0 con 0x7f1f0c141360 2024-03-18T15:33:51.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.429+0000 7f1f1467d640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f1f0c06c0e0 con 0x7f1f0c147bd0 2024-03-18T15:33:51.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.429+0000 7f1f1467d640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f1f0c0719b0 con 0x7f1f0c06b4e0 2024-03-18T15:33:51.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.429+0000 7f1f12419640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1f0c06b4e0 0x7f1f0c0702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:51.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.429+0000 7f1f11c18640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f1f0c141360 0x7f1f0c1474c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.5:39214/0 (socket says 172.21.15.5:39214) 2024-03-18T15:33:51.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.429+0000 7f1f12c1a640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1f0c147bd0 0x7f1f0c1417e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:51.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.429+0000 7f1f11c18640 1 -- 172.21.15.5:0/3697795035 learned_addr learned my addr 172.21.15.5:0/3697795035 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:33:51.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.429+0000 7f1f11417640 1 -- 172.21.15.5:0/3697795035 <== mon.1 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3488105906 0 0) 0x7f1f0c06b8c0 con 0x7f1f0c141360 2024-03-18T15:33:51.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.429+0000 7f1f11417640 1 -- 172.21.15.5:0/3697795035 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ef8003530 con 0x7f1f0c141360 2024-03-18T15:33:51.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.429+0000 7f1f12c1a640 1 -- 172.21.15.5:0/3697795035 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1f0c06b4e0 msgr2=0x7f1f0c0702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:51.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.429+0000 7f1f12c1a640 1 --2- 172.21.15.5:0/3697795035 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1f0c06b4e0 0x7f1f0c0702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:51.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.429+0000 7f1f12c1a640 1 -- 172.21.15.5:0/3697795035 >> v1:172.21.15.80:6789/0 conn(0x7f1f0c141360 legacy=0x7f1f0c1474c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:51.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.429+0000 7f1f12c1a640 1 -- 172.21.15.5:0/3697795035 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f0c06e800 con 0x7f1f0c147bd0 2024-03-18T15:33:51.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.429+0000 7f1f12419640 1 --2- 172.21.15.5:0/3697795035 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1f0c06b4e0 0x7f1f0c0702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:33:51.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.433+0000 7f1f12c1a640 1 --2- 172.21.15.5:0/3697795035 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f0c147bd0 0x7f1f0c1417e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1f040097c0 tx=0x7f1f0401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e7d44f0aa7f0aa8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:51.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.433+0000 7f1f11417640 1 -- 172.21.15.5:0/3697795035 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f040b0070 con 0x7f1f0c147bd0 2024-03-18T15:33:51.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.433+0000 7f1f11417640 1 -- 172.21.15.5:0/3697795035 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f040ab600 con 0x7f1f0c147bd0 2024-03-18T15:33:51.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.433+0000 7f1f11417640 1 -- 172.21.15.5:0/3697795035 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f040ab9b0 con 0x7f1f0c147bd0 2024-03-18T15:33:51.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.433+0000 7f1f1467d640 1 -- 172.21.15.5:0/3697795035 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f0c147bd0 msgr2=0x7f1f0c1417e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:51.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.433+0000 7f1f1467d640 1 --2- 172.21.15.5:0/3697795035 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f0c147bd0 0x7f1f0c1417e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1f040097c0 tx=0x7f1f0401c410 comp rx=0 tx=0).stop 2024-03-18T15:33:51.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.433+0000 7f1f1467d640 1 -- 172.21.15.5:0/3697795035 shutdown_connections 2024-03-18T15:33:51.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.433+0000 7f1f1467d640 1 --2- 172.21.15.5:0/3697795035 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f0c147bd0 0x7f1f0c1417e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:51.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.433+0000 7f1f1467d640 1 --2- 172.21.15.5:0/3697795035 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1f0c06b4e0 0x7f1f0c0702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:51.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.433+0000 7f1f1467d640 1 -- 172.21.15.5:0/3697795035 >> 172.21.15.5:0/3697795035 conn(0x7f1f0c0b7a80 msgr2=0x7f1f0c0b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:51.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.433+0000 7f1f1467d640 1 -- 172.21.15.5:0/3697795035 shutdown_connections 2024-03-18T15:33:51.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.433+0000 7f1f1467d640 1 -- 172.21.15.5:0/3697795035 wait complete. 2024-03-18T15:33:51.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.433+0000 7f1f1467d640 1 Processor -- start 2024-03-18T15:33:51.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.433+0000 7f1f1467d640 1 -- start start 2024-03-18T15:33:51.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.433+0000 7f1f1467d640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1f0c06b4e0 0x7f1f0c140050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:51.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.433+0000 7f1f1467d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f0c141360 0x7f1f0c138070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:51.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.433+0000 7f1f1467d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f0c136d70 0x7f1f0c137220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:51.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.433+0000 7f1f1467d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1f0c154910 con 0x7f1f0c136d70 2024-03-18T15:33:51.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.433+0000 7f1f1467d640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1f0c154790 con 0x7f1f0c141360 2024-03-18T15:33:51.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.433+0000 7f1f1467d640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f1f0c154a90 con 0x7f1f0c06b4e0 2024-03-18T15:33:51.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.433+0000 7f1f12c1a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f0c136d70 0x7f1f0c137220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:51.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.433+0000 7f1f12c1a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f0c136d70 0x7f1f0c137220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42896/0 (socket says 172.21.15.5:42896) 2024-03-18T15:33:51.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.433+0000 7f1f12c1a640 1 -- 172.21.15.5:0/1070068210 learned_addr learned my addr 172.21.15.5:0/1070068210 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:51.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.433+0000 7f1f11c18640 1 --2- 172.21.15.5:0/1070068210 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f0c141360 0x7f1f0c138070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:51.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.433+0000 7f1f12419640 1 --2- 172.21.15.5:0/1070068210 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1f0c06b4e0 0x7f1f0c140050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:51.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.437+0000 7f1f12c1a640 1 -- 172.21.15.5:0/1070068210 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1f0c06b4e0 msgr2=0x7f1f0c140050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:51.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.437+0000 7f1f12c1a640 1 --2- 172.21.15.5:0/1070068210 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1f0c06b4e0 0x7f1f0c140050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:51.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.437+0000 7f1f12c1a640 1 -- 172.21.15.5:0/1070068210 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f0c141360 msgr2=0x7f1f0c138070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:51.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.437+0000 7f1f12c1a640 1 --2- 172.21.15.5:0/1070068210 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f0c141360 0x7f1f0c138070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:51.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.437+0000 7f1f12c1a640 1 -- 172.21.15.5:0/1070068210 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f0c1339a0 con 0x7f1f0c136d70 2024-03-18T15:33:51.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.437+0000 7f1f12c1a640 1 --2- 172.21.15.5:0/1070068210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f0c136d70 0x7f1f0c137220 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1f040ae040 tx=0x7f1f040191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:51.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.437+0000 7f1f02ffd640 1 -- 172.21.15.5:0/1070068210 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f040b0040 con 0x7f1f0c136d70 2024-03-18T15:33:51.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.437+0000 7f1f02ffd640 1 -- 172.21.15.5:0/1070068210 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f040ab9b0 con 0x7f1f0c136d70 2024-03-18T15:33:51.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.437+0000 7f1f1467d640 1 -- 172.21.15.5:0/1070068210 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f0c1389b0 con 0x7f1f0c136d70 2024-03-18T15:33:51.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.437+0000 7f1f02ffd640 1 -- 172.21.15.5:0/1070068210 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f040abd60 con 0x7f1f0c136d70 2024-03-18T15:33:51.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.437+0000 7f1f1467d640 1 -- 172.21.15.5:0/1070068210 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f0c136850 con 0x7f1f0c136d70 2024-03-18T15:33:51.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.437+0000 7f1f1467d640 1 -- 172.21.15.5:0/1070068210 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f0c06c310 con 0x7f1f0c136d70 2024-03-18T15:33:51.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.437+0000 7f1f02ffd640 1 -- 172.21.15.5:0/1070068210 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f1f040ab480 con 0x7f1f0c136d70 2024-03-18T15:33:51.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.437+0000 7f1f02ffd640 1 --2- 172.21.15.5:0/1070068210 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f1ef0067660 0x7f1ef0069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:51.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.437+0000 7f1f02ffd640 1 -- 172.21.15.5:0/1070068210 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1f0411f2c0 con 0x7f1f0c136d70 2024-03-18T15:33:51.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.437+0000 7f1f12419640 1 --2- 172.21.15.5:0/1070068210 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f1ef0067660 0x7f1ef0069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:51.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.437+0000 7f1f12419640 1 --2- 172.21.15.5:0/1070068210 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f1ef0067660 0x7f1ef0069b20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1efc009a50 tx=0x7f1efc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:51.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.441+0000 7f1f02ffd640 1 -- 172.21.15.5:0/1070068210 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f1f04016090 con 0x7f1f0c136d70 2024-03-18T15:33:51.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.593+0000 7f1f1467d640 1 -- 172.21.15.5:0/1070068210 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1f0c04f550 con 0x7f1f0c136d70 2024-03-18T15:33:51.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.593+0000 7f1f02ffd640 1 -- 172.21.15.5:0/1070068210 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f1f040effd0 con 0x7f1f0c136d70 2024-03-18T15:33:51.596 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-03-18T15:33:51.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.597+0000 7f1f1467d640 1 -- 172.21.15.5:0/1070068210 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f1ef0067660 msgr2=0x7f1ef0069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:51.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.597+0000 7f1f1467d640 1 --2- 172.21.15.5:0/1070068210 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f1ef0067660 0x7f1ef0069b20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1efc009a50 tx=0x7f1efc008040 comp rx=0 tx=0).stop 2024-03-18T15:33:51.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.597+0000 7f1f1467d640 1 -- 172.21.15.5:0/1070068210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f0c136d70 msgr2=0x7f1f0c137220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:51.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.597+0000 7f1f1467d640 1 --2- 172.21.15.5:0/1070068210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f0c136d70 0x7f1f0c137220 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1f040ae040 tx=0x7f1f040191b0 comp rx=0 tx=0).stop 2024-03-18T15:33:51.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.597+0000 7f1f1467d640 1 -- 172.21.15.5:0/1070068210 shutdown_connections 2024-03-18T15:33:51.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.597+0000 7f1f1467d640 1 --2- 172.21.15.5:0/1070068210 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f1ef0067660 0x7f1ef0069b20 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:51.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.597+0000 7f1f1467d640 1 --2- 172.21.15.5:0/1070068210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f0c136d70 0x7f1f0c137220 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:51.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.597+0000 7f1f1467d640 1 --2- 172.21.15.5:0/1070068210 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f0c141360 0x7f1f0c138070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:51.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.597+0000 7f1f1467d640 1 --2- 172.21.15.5:0/1070068210 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1f0c06b4e0 0x7f1f0c140050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:51.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.597+0000 7f1f1467d640 1 -- 172.21.15.5:0/1070068210 >> 172.21.15.5:0/1070068210 conn(0x7f1f0c0b7a80 msgr2=0x7f1f0c13e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:51.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.597+0000 7f1f1467d640 1 -- 172.21.15.5:0/1070068210 shutdown_connections 2024-03-18T15:33:51.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.597+0000 7f1f1467d640 1 -- 172.21.15.5:0/1070068210 wait complete. 2024-03-18T15:33:51.614 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:33:51.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.721+0000 7f574ab7e640 1 Processor -- start 2024-03-18T15:33:51.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.721+0000 7f574ab7e640 1 -- start start 2024-03-18T15:33:51.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.725+0000 7f574ab7e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f57440b2db0 0x7f574406d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:51.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.725+0000 7f574ab7e640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57440afb70 con 0x7f5744072b40 2024-03-18T15:33:51.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.725+0000 7f574ab7e640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57440b4a20 con 0x7f57440af790 2024-03-18T15:33:51.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.725+0000 7f574ab7e640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f574406d5b0 con 0x7f57440b2db0 2024-03-18T15:33:51.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.725+0000 7f574911b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f57440b2db0 0x7f574406d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:51.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.725+0000 7f5743fff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f5744072b40 0x7f57440b26a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46878/0 (socket says 172.21.15.5:46878) 2024-03-18T15:33:51.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.725+0000 7f5743fff640 1 -- 172.21.15.5:0/3256592115 learned_addr learned my addr 172.21.15.5:0/3256592115 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:33:51.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.725+0000 7f574911b640 1 -- 172.21.15.5:0/3256592115 >> v1:172.21.15.80:6789/0 conn(0x7f57440af790 legacy=0x7f5744147780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:51.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.725+0000 7f574911b640 1 -- 172.21.15.5:0/3256592115 >> v1:172.21.15.5:6789/0 conn(0x7f5744072b40 legacy=0x7f57440b26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:51.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.725+0000 7f574911b640 1 -- 172.21.15.5:0/3256592115 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57440b4d00 con 0x7f57440b2db0 2024-03-18T15:33:51.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.725+0000 7f574911b640 1 --2- 172.21.15.5:0/3256592115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57440b2db0 0x7f574406d070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f573c009d90 tx=0x7f573c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68e8eecd85a283 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:51.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.725+0000 7f57437fe640 1 -- 172.21.15.5:0/3256592115 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573c0b2070 con 0x7f57440b2db0 2024-03-18T15:33:51.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.725+0000 7f57437fe640 1 -- 172.21.15.5:0/3256592115 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f573c0a57a0 con 0x7f57440b2db0 2024-03-18T15:33:51.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.725+0000 7f57437fe640 1 -- 172.21.15.5:0/3256592115 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573c0a5b70 con 0x7f57440b2db0 2024-03-18T15:33:51.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.725+0000 7f574ab7e640 1 -- 172.21.15.5:0/3256592115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57440b2db0 msgr2=0x7f574406d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:51.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.725+0000 7f574ab7e640 1 --2- 172.21.15.5:0/3256592115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57440b2db0 0x7f574406d070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f573c009d90 tx=0x7f573c01c8e0 comp rx=0 tx=0).stop 2024-03-18T15:33:51.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.725+0000 7f574ab7e640 1 -- 172.21.15.5:0/3256592115 shutdown_connections 2024-03-18T15:33:51.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.725+0000 7f574ab7e640 1 --2- 172.21.15.5:0/3256592115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57440b2db0 0x7f574406d070 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:51.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.725+0000 7f574ab7e640 1 -- 172.21.15.5:0/3256592115 >> 172.21.15.5:0/3256592115 conn(0x7f57440b5670 msgr2=0x7f57440b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:51.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.725+0000 7f574ab7e640 1 -- 172.21.15.5:0/3256592115 shutdown_connections 2024-03-18T15:33:51.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.725+0000 7f574ab7e640 1 -- 172.21.15.5:0/3256592115 wait complete. 2024-03-18T15:33:51.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.729+0000 7f574ab7e640 1 Processor -- start 2024-03-18T15:33:51.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.729+0000 7f574ab7e640 1 -- start start 2024-03-18T15:33:51.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.729+0000 7f574ab7e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5744072b40 0x7f5744140bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:51.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.729+0000 7f574ab7e640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f57440af790 0x7f574414cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:51.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.729+0000 7f574ab7e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57440b2db0 0x7f57441e34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:51.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.729+0000 7f574ab7e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5744152ac0 con 0x7f5744072b40 2024-03-18T15:33:51.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.729+0000 7f574ab7e640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f5744152940 con 0x7f57440b2db0 2024-03-18T15:33:51.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.729+0000 7f574ab7e640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f5744152c40 con 0x7f57440af790 2024-03-18T15:33:51.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.729+0000 7f574891a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5744072b40 0x7f5744140bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:51.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.729+0000 7f574891a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5744072b40 0x7f5744140bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42904/0 (socket says 172.21.15.5:42904) 2024-03-18T15:33:51.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.729+0000 7f574891a640 1 -- 172.21.15.5:0/2722015258 learned_addr learned my addr 172.21.15.5:0/2722015258 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:51.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.729+0000 7f574911b640 1 --2- 172.21.15.5:0/2722015258 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57440b2db0 0x7f57441e34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:51.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.729+0000 7f5743fff640 1 --2- 172.21.15.5:0/2722015258 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f57440af790 0x7f574414cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:51.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.729+0000 7f574891a640 1 -- 172.21.15.5:0/2722015258 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f57440af790 msgr2=0x7f574414cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:51.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.729+0000 7f574891a640 1 --2- 172.21.15.5:0/2722015258 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f57440af790 0x7f574414cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:51.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.729+0000 7f574891a640 1 -- 172.21.15.5:0/2722015258 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57440b2db0 msgr2=0x7f57441e34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:51.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.729+0000 7f574891a640 1 --2- 172.21.15.5:0/2722015258 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57440b2db0 0x7f57441e34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:51.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.729+0000 7f574891a640 1 -- 172.21.15.5:0/2722015258 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57441e0470 con 0x7f5744072b40 2024-03-18T15:33:51.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.729+0000 7f574911b640 1 --2- 172.21.15.5:0/2722015258 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57440b2db0 0x7f57441e34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:33:51.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.729+0000 7f574891a640 1 --2- 172.21.15.5:0/2722015258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5744072b40 0x7f5744140bd0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f573800c450 tx=0x7f573800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:51.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.729+0000 7f57417fa640 1 -- 172.21.15.5:0/2722015258 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5738016020 con 0x7f5744072b40 2024-03-18T15:33:51.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.729+0000 7f57417fa640 1 -- 172.21.15.5:0/2722015258 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57380052d0 con 0x7f5744072b40 2024-03-18T15:33:51.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.729+0000 7f574ab7e640 1 -- 172.21.15.5:0/2722015258 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57441e0710 con 0x7f5744072b40 2024-03-18T15:33:51.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.729+0000 7f57417fa640 1 -- 172.21.15.5:0/2722015258 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5738007260 con 0x7f5744072b40 2024-03-18T15:33:51.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.729+0000 7f574ab7e640 1 -- 172.21.15.5:0/2722015258 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57441418c0 con 0x7f5744072b40 2024-03-18T15:33:51.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.729+0000 7f574ab7e640 1 -- 172.21.15.5:0/2722015258 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f574404f550 con 0x7f5744072b40 2024-03-18T15:33:51.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.729+0000 7f57417fa640 1 -- 172.21.15.5:0/2722015258 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f5738007400 con 0x7f5744072b40 2024-03-18T15:33:51.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.733+0000 7f57417fa640 1 --2- 172.21.15.5:0/2722015258 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5730067660 0x7f5730069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:51.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.733+0000 7f57417fa640 1 -- 172.21.15.5:0/2722015258 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f5738085220 con 0x7f5744072b40 2024-03-18T15:33:51.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.733+0000 7f5743fff640 1 --2- 172.21.15.5:0/2722015258 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5730067660 0x7f5730069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:51.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.733+0000 7f5743fff640 1 --2- 172.21.15.5:0/2722015258 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5730067660 0x7f5730069b20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f57440723f0 tx=0x7f5734009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:51.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.737+0000 7f57417fa640 1 -- 172.21.15.5:0/2722015258 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f5738020020 con 0x7f5744072b40 2024-03-18T15:33:51.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.921+0000 7f574ab7e640 1 -- 172.21.15.5:0/2722015258 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f574413e780 con 0x7f5744072b40 2024-03-18T15:33:51.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.921+0000 7f57417fa640 1 -- 172.21.15.5:0/2722015258 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+3521 (secure 0 0 0) 0x7f5738005480 con 0x7f5744072b40 2024-03-18T15:33:51.925 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:33:51.925 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/629589896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":629589896},{"type":"v1","addr":"172.21.15.5:6835","nonce":629589896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-18T15:33:42.338926+0000","modified":"2024-03-18T15:33:45.376646+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6833/1236603350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":1236603350},{"type":"v1","addr":"172.21.15.80:6833","nonce":1236603350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4272,"qdb_cluster":[4272]},"id":1}]} 2024-03-18T15:33:51.925 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 8 2024-03-18T15:33:51.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.925+0000 7f574ab7e640 1 -- 172.21.15.5:0/2722015258 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5730067660 msgr2=0x7f5730069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:51.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.925+0000 7f574ab7e640 1 --2- 172.21.15.5:0/2722015258 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5730067660 0x7f5730069b20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f57440723f0 tx=0x7f5734009040 comp rx=0 tx=0).stop 2024-03-18T15:33:51.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.925+0000 7f574ab7e640 1 -- 172.21.15.5:0/2722015258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5744072b40 msgr2=0x7f5744140bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:51.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.925+0000 7f574ab7e640 1 --2- 172.21.15.5:0/2722015258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5744072b40 0x7f5744140bd0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f573800c450 tx=0x7f573800c910 comp rx=0 tx=0).stop 2024-03-18T15:33:51.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.925+0000 7f574ab7e640 1 -- 172.21.15.5:0/2722015258 shutdown_connections 2024-03-18T15:33:51.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.925+0000 7f574ab7e640 1 --2- 172.21.15.5:0/2722015258 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5730067660 0x7f5730069b20 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:51.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.925+0000 7f574ab7e640 1 --2- 172.21.15.5:0/2722015258 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57440b2db0 0x7f57441e34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:51.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.925+0000 7f574ab7e640 1 --2- 172.21.15.5:0/2722015258 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f57440af790 0x7f574414cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:51.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.925+0000 7f574ab7e640 1 --2- 172.21.15.5:0/2722015258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5744072b40 0x7f5744140bd0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:51.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.925+0000 7f574ab7e640 1 -- 172.21.15.5:0/2722015258 >> 172.21.15.5:0/2722015258 conn(0x7f57440b5670 msgr2=0x7f57440b5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:51.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.925+0000 7f574ab7e640 1 -- 172.21.15.5:0/2722015258 shutdown_connections 2024-03-18T15:33:51.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:51.925+0000 7f574ab7e640 1 -- 172.21.15.5:0/2722015258 wait complete. 2024-03-18T15:33:51.942 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:33:52.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.061+0000 7fa767841640 1 Processor -- start 2024-03-18T15:33:52.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.061+0000 7fa767841640 1 -- start start 2024-03-18T15:33:52.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.061+0000 7fa767841640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa76006d2d0 0x7fa7600b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:52.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.061+0000 7fa767841640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7600b0ea0 con 0x7fa76013f6a0 2024-03-18T15:33:52.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.061+0000 7fa767841640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa76006d6b0 con 0x7fa76013f260 2024-03-18T15:33:52.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.061+0000 7fa767841640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fa7600728c0 con 0x7fa76006d2d0 2024-03-18T15:33:52.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.061+0000 7fa765dde640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa76013f6a0 0x7fa760145ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46888/0 (socket says 172.21.15.5:46888) 2024-03-18T15:33:52.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.061+0000 7fa7655dd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa76006d2d0 0x7fa7600b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:52.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.061+0000 7fa765dde640 1 -- 172.21.15.5:0/500204992 learned_addr learned my addr 172.21.15.5:0/500204992 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:33:52.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.061+0000 7fa7655dd640 1 -- 172.21.15.5:0/500204992 >> v1:172.21.15.80:6789/0 conn(0x7fa76013f260 legacy=0x7fa7601453a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:52.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.061+0000 7fa7655dd640 1 -- 172.21.15.5:0/500204992 >> v1:172.21.15.5:6789/0 conn(0x7fa76013f6a0 legacy=0x7fa760145ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:52.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.061+0000 7fa7655dd640 1 -- 172.21.15.5:0/500204992 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa76013faf0 con 0x7fa76006d2d0 2024-03-18T15:33:52.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.061+0000 7fa757fff640 1 -- 172.21.15.5:0/500204992 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 473705047 0 0) 0x7fa7600b0ea0 con 0x7fa76013f6a0 2024-03-18T15:33:52.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.061+0000 7fa7655dd640 1 --2- 172.21.15.5:0/500204992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa76006d2d0 0x7fa7600b23a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa75c0097c0 tx=0x7fa75c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6437c49c01bc5823 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:52.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.061+0000 7fa757fff640 1 -- 172.21.15.5:0/500204992 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75c0b1070 con 0x7fa76006d2d0 2024-03-18T15:33:52.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.061+0000 7fa757fff640 1 -- 172.21.15.5:0/500204992 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa75c0a8050 con 0x7fa76006d2d0 2024-03-18T15:33:52.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.061+0000 7fa757fff640 1 -- 172.21.15.5:0/500204992 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75c0ac860 con 0x7fa76006d2d0 2024-03-18T15:33:52.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.065+0000 7fa767841640 1 -- 172.21.15.5:0/500204992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa76006d2d0 msgr2=0x7fa7600b23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:52.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.065+0000 7fa767841640 1 --2- 172.21.15.5:0/500204992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa76006d2d0 0x7fa7600b23a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa75c0097c0 tx=0x7fa75c01c3e0 comp rx=0 tx=0).stop 2024-03-18T15:33:52.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.065+0000 7fa767841640 1 -- 172.21.15.5:0/500204992 shutdown_connections 2024-03-18T15:33:52.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.065+0000 7fa767841640 1 --2- 172.21.15.5:0/500204992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa76006d2d0 0x7fa7600b23a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:52.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.065+0000 7fa767841640 1 -- 172.21.15.5:0/500204992 >> 172.21.15.5:0/500204992 conn(0x7fa7600b5670 msgr2=0x7fa7600b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:52.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.065+0000 7fa767841640 1 -- 172.21.15.5:0/500204992 shutdown_connections 2024-03-18T15:33:52.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.065+0000 7fa767841640 1 -- 172.21.15.5:0/500204992 wait complete. 2024-03-18T15:33:52.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.065+0000 7fa767841640 1 Processor -- start 2024-03-18T15:33:52.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.065+0000 7fa767841640 1 -- start start 2024-03-18T15:33:52.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.065+0000 7fa767841640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa76006d2d0 0x7fa7600ae1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:52.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.065+0000 7fa767841640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa76013f260 0x7fa7601df900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:52.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.065+0000 7fa767841640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa76013f6a0 0x7fa7601dfe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:52.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.065+0000 7fa767841640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa7600b01f0 con 0x7fa76013f6a0 2024-03-18T15:33:52.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.065+0000 7fa767841640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa7600b0070 con 0x7fa76013f260 2024-03-18T15:33:52.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.065+0000 7fa767841640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fa7600b07e0 con 0x7fa76006d2d0 2024-03-18T15:33:52.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.065+0000 7fa765dde640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa76013f6a0 0x7fa7601dfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:52.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.065+0000 7fa765dde640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa76013f6a0 0x7fa7601dfe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42916/0 (socket says 172.21.15.5:42916) 2024-03-18T15:33:52.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.065+0000 7fa765dde640 1 -- 172.21.15.5:0/2836514152 learned_addr learned my addr 172.21.15.5:0/2836514152 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:52.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.065+0000 7fa7655dd640 1 --2- 172.21.15.5:0/2836514152 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa76006d2d0 0x7fa7600ae1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:52.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.065+0000 7fa764ddc640 1 --2- 172.21.15.5:0/2836514152 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa76013f260 0x7fa7601df900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:52.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.065+0000 7fa765dde640 1 -- 172.21.15.5:0/2836514152 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa76006d2d0 msgr2=0x7fa7600ae1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:52.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.065+0000 7fa765dde640 1 --2- 172.21.15.5:0/2836514152 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa76006d2d0 0x7fa7600ae1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:52.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.065+0000 7fa765dde640 1 -- 172.21.15.5:0/2836514152 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa76013f260 msgr2=0x7fa7601df900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:52.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.065+0000 7fa765dde640 1 --2- 172.21.15.5:0/2836514152 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa76013f260 0x7fa7601df900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:52.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.065+0000 7fa765dde640 1 -- 172.21.15.5:0/2836514152 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa760140e40 con 0x7fa76013f6a0 2024-03-18T15:33:52.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.065+0000 7fa765dde640 1 --2- 172.21.15.5:0/2836514152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa76013f6a0 0x7fa7601dfe40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa75800c700 tx=0x7fa75800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:52.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.065+0000 7fa755ffb640 1 -- 172.21.15.5:0/2836514152 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa758018070 con 0x7fa76013f6a0 2024-03-18T15:33:52.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.065+0000 7fa755ffb640 1 -- 172.21.15.5:0/2836514152 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa75800ce10 con 0x7fa76013f6a0 2024-03-18T15:33:52.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.065+0000 7fa767841640 1 -- 172.21.15.5:0/2836514152 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa760071f40 con 0x7fa76013f6a0 2024-03-18T15:33:52.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.065+0000 7fa755ffb640 1 -- 172.21.15.5:0/2836514152 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75800a800 con 0x7fa76013f6a0 2024-03-18T15:33:52.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.065+0000 7fa767841640 1 -- 172.21.15.5:0/2836514152 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7600aef60 con 0x7fa76013f6a0 2024-03-18T15:33:52.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.069+0000 7fa767841640 1 -- 172.21.15.5:0/2836514152 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa76004f550 con 0x7fa76013f6a0 2024-03-18T15:33:52.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.069+0000 7fa755ffb640 1 -- 172.21.15.5:0/2836514152 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fa75801b070 con 0x7fa76013f6a0 2024-03-18T15:33:52.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.069+0000 7fa755ffb640 1 --2- 172.21.15.5:0/2836514152 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa744067660 0x7fa744069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:52.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.069+0000 7fa755ffb640 1 -- 172.21.15.5:0/2836514152 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa758084d60 con 0x7fa76013f6a0 2024-03-18T15:33:52.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.069+0000 7fa7655dd640 1 --2- 172.21.15.5:0/2836514152 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa744067660 0x7fa744069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:52.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.069+0000 7fa7655dd640 1 --2- 172.21.15.5:0/2836514152 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa744067660 0x7fa744069b20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa75c0ae000 tx=0x7fa75c0a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:52.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.073+0000 7fa755ffb640 1 -- 172.21.15.5:0/2836514152 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fa758055a70 con 0x7fa76013f6a0 2024-03-18T15:33:52.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.229+0000 7fa767841640 1 -- 172.21.15.5:0/2836514152 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa7600af220 con 0x7fa76013f6a0 2024-03-18T15:33:52.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.229+0000 7fa755ffb640 1 -- 172.21.15.5:0/2836514152 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16449 (secure 0 0 0) 0x7fa758055370 con 0x7fa76013f6a0 2024-03-18T15:33:52.233 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:33:52.233 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":21,"fsid":"ef54fe03-5aa7-4eb2-9b1e-66c9cc345b94","created":"2024-03-18T15:33:18.086614+0000","modified":"2024-03-18T15:33:43.984050+0000","last_up_change":"2024-03-18T15:33:29.229911+0000","last_in_change":"2024-03-18T15:33:20.323840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:28.963186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T15:33:40.533646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T15:33:41.323519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df1c754b-af59-4d16-8919-fbb6386b3c48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6804","nonce":976264754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6808","nonce":976264754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6815","nonce":976264754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6811","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6813","nonce":976264754}]},"public_addr":"172.21.15.5:6804/976264754","cluster_addr":"172.21.15.5:6808/976264754","heartbeat_back_addr":"172.21.15.5:6815/976264754","heartbeat_front_addr":"172.21.15.5:6813/976264754","state":["exists","up"]},{"osd":1,"uuid":"b9a08b5e-8af4-4144-bdba-d69ed1ac627a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6801","nonce":3901761126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6803","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6805","nonce":3901761126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6812","nonce":3901761126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6807","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6809","nonce":3901761126}]},"public_addr":"172.21.15.5:6801/3901761126","cluster_addr":"172.21.15.5:6805/3901761126","heartbeat_back_addr":"172.21.15.5:6812/3901761126","heartbeat_front_addr":"172.21.15.5:6809/3901761126","state":["exists","up"]},{"osd":2,"uuid":"c3e8bb76-e382-439d-b380-62ef92f1255a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6817","nonce":502968448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6819","nonce":502968448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6823","nonce":502968448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6821","nonce":502968448}]},"public_addr":"172.21.15.5:6817/502968448","cluster_addr":"172.21.15.5:6819/502968448","heartbeat_back_addr":"172.21.15.5:6823/502968448","heartbeat_front_addr":"172.21.15.5:6821/502968448","state":["exists","up"]},{"osd":3,"uuid":"e8c94625-f2cd-4eba-b53d-2e7356e1c02a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6825","nonce":2425932735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6827","nonce":2425932735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6831","nonce":2425932735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6829","nonce":2425932735}]},"public_addr":"172.21.15.5:6825/2425932735","cluster_addr":"172.21.15.5:6827/2425932735","heartbeat_back_addr":"172.21.15.5:6831/2425932735","heartbeat_front_addr":"172.21.15.5:6829/2425932735","state":["exists","up"]},{"osd":4,"uuid":"0099dd52-6265-4d32-82e4-3ef16e73add0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6817","nonce":976514285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6819","nonce":976514285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6823","nonce":976514285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6821","nonce":976514285}]},"public_addr":"172.21.15.80:6817/976514285","cluster_addr":"172.21.15.80:6819/976514285","heartbeat_back_addr":"172.21.15.80:6823/976514285","heartbeat_front_addr":"172.21.15.80:6821/976514285","state":["exists","up"]},{"osd":5,"uuid":"ed10c84f-8c39-41cc-9b64-7c482a94a48e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6825","nonce":627153447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6827","nonce":627153447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6831","nonce":627153447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6829","nonce":627153447}]},"public_addr":"172.21.15.80:6825/627153447","cluster_addr":"172.21.15.80:6827/627153447","heartbeat_back_addr":"172.21.15.80:6831/627153447","heartbeat_front_addr":"172.21.15.80:6829/627153447","state":["exists","up"]},{"osd":6,"uuid":"c7a3538a-7048-4fba-8781-b440f03c25db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6803","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6805","nonce":488342527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6807","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6809","nonce":488342527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6815","nonce":488342527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6811","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6813","nonce":488342527}]},"public_addr":"172.21.15.80:6805/488342527","cluster_addr":"172.21.15.80:6809/488342527","heartbeat_back_addr":"172.21.15.80:6815/488342527","heartbeat_front_addr":"172.21.15.80:6813/488342527","state":["exists","up"]},{"osd":7,"uuid":"f9418104-3a97-4110-bc3a-1f8151d0f757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6801","nonce":982831666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6804","nonce":982831666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6812","nonce":982831666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6808","nonce":982831666}]},"public_addr":"172.21.15.80:6801/982831666","cluster_addr":"172.21.15.80:6804/982831666","heartbeat_back_addr":"172.21.15.80:6812/982831666","heartbeat_front_addr":"172.21.15.80:6808/982831666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.970828+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.858116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.987556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.864603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.756293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.591502+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:33:52.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.233+0000 7fa767841640 1 -- 172.21.15.5:0/2836514152 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa744067660 msgr2=0x7fa744069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:52.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.233+0000 7fa767841640 1 --2- 172.21.15.5:0/2836514152 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa744067660 0x7fa744069b20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa75c0ae000 tx=0x7fa75c0a7000 comp rx=0 tx=0).stop 2024-03-18T15:33:52.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.233+0000 7fa767841640 1 -- 172.21.15.5:0/2836514152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa76013f6a0 msgr2=0x7fa7601dfe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:52.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.233+0000 7fa767841640 1 --2- 172.21.15.5:0/2836514152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa76013f6a0 0x7fa7601dfe40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa75800c700 tx=0x7fa75800cbc0 comp rx=0 tx=0).stop 2024-03-18T15:33:52.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.233+0000 7fa767841640 1 -- 172.21.15.5:0/2836514152 shutdown_connections 2024-03-18T15:33:52.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.233+0000 7fa767841640 1 --2- 172.21.15.5:0/2836514152 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa744067660 0x7fa744069b20 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:52.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.233+0000 7fa767841640 1 --2- 172.21.15.5:0/2836514152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa76013f6a0 0x7fa7601dfe40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:52.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.233+0000 7fa767841640 1 --2- 172.21.15.5:0/2836514152 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa76013f260 0x7fa7601df900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:52.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.233+0000 7fa767841640 1 --2- 172.21.15.5:0/2836514152 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa76006d2d0 0x7fa7600ae1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:52.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.233+0000 7fa767841640 1 -- 172.21.15.5:0/2836514152 >> 172.21.15.5:0/2836514152 conn(0x7fa7600b5670 msgr2=0x7fa76013cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:52.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.233+0000 7fa767841640 1 -- 172.21.15.5:0/2836514152 shutdown_connections 2024-03-18T15:33:52.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.233+0000 7fa767841640 1 -- 172.21.15.5:0/2836514152 wait complete. 2024-03-18T15:33:52.250 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T15:33:28.963186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-18T15:33:40.533646+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-18T15:33:41.323519+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T15:33:52.251 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T15:33:52.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.365+0000 7f78c9630640 1 Processor -- start 2024-03-18T15:33:52.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.365+0000 7f78c9630640 1 -- start start 2024-03-18T15:33:52.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.369+0000 7f78c9630640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f78c4148950 0x7f78c40b6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:52.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.369+0000 7f78c9630640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f78c40b65c0 0x7f78c40b45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:52.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.369+0000 7f78c9630640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78c40b5d90 con 0x7f78c40b1c30 2024-03-18T15:33:52.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.369+0000 7f78c9630640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f78c4149440 con 0x7f78c40b65c0 2024-03-18T15:33:52.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.369+0000 7f78c9630640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f78c40b6070 con 0x7f78c4148950 2024-03-18T15:33:52.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.369+0000 7f78c37fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f78c40b65c0 0x7f78c40b45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:52.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.369+0000 7f78c37fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f78c40b65c0 0x7f78c40b45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42920/0 (socket says 172.21.15.5:42920) 2024-03-18T15:33:52.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.369+0000 7f78c37fe640 1 -- 172.21.15.5:0/2647845449 learned_addr learned my addr 172.21.15.5:0/2647845449 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:52.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.369+0000 7f78c2ffd640 1 --2- 172.21.15.5:0/2647845449 >> v2:172.21.15.80:3300/0 conn(0x7f78c4148950 0x7f78c40b6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:52.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.369+0000 7f78c37fe640 1 -- 172.21.15.5:0/2647845449 >> v2:172.21.15.80:3300/0 conn(0x7f78c4148950 msgr2=0x7f78c40b6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:52.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.369+0000 7f78c37fe640 1 --2- 172.21.15.5:0/2647845449 >> v2:172.21.15.80:3300/0 conn(0x7f78c4148950 0x7f78c40b6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:52.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.369+0000 7f78c37fe640 1 -- 172.21.15.5:0/2647845449 >> v1:172.21.15.80:6789/0 conn(0x7f78c40b1c30 legacy=0x7f78c4148d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:52.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.369+0000 7f78c37fe640 1 -- 172.21.15.5:0/2647845449 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c40b56b0 con 0x7f78c40b65c0 2024-03-18T15:33:52.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.369+0000 7f78c37fe640 1 --2- 172.21.15.5:0/2647845449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78c40b65c0 0x7f78c40b45a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f78c40710b0 tx=0x7f78bc01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dec06ddf5bdc9615 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:52.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.373+0000 7f78c1ffb640 1 -- 172.21.15.5:0/2647845449 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78bc0b0070 con 0x7f78c40b65c0 2024-03-18T15:33:52.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.373+0000 7f78c1ffb640 1 -- 172.21.15.5:0/2647845449 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78bc0a4e50 con 0x7f78c40b65c0 2024-03-18T15:33:52.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.373+0000 7f78c1ffb640 1 -- 172.21.15.5:0/2647845449 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78bc0ab820 con 0x7f78c40b65c0 2024-03-18T15:33:52.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.373+0000 7f78c9630640 1 -- 172.21.15.5:0/2647845449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78c40b65c0 msgr2=0x7f78c40b45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:52.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.373+0000 7f78c9630640 1 --2- 172.21.15.5:0/2647845449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78c40b65c0 0x7f78c40b45a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f78c40710b0 tx=0x7f78bc01c3c0 comp rx=0 tx=0).stop 2024-03-18T15:33:52.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.373+0000 7f78c9630640 1 -- 172.21.15.5:0/2647845449 shutdown_connections 2024-03-18T15:33:52.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.373+0000 7f78c9630640 1 --2- 172.21.15.5:0/2647845449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78c40b65c0 0x7f78c40b45a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:52.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.373+0000 7f78c9630640 1 --2- 172.21.15.5:0/2647845449 >> v2:172.21.15.80:3300/0 conn(0x7f78c4148950 0x7f78c40b6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:52.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.373+0000 7f78c9630640 1 -- 172.21.15.5:0/2647845449 >> 172.21.15.5:0/2647845449 conn(0x7f78c40bacb0 msgr2=0x7f78c40bb0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:52.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.373+0000 7f78c9630640 1 -- 172.21.15.5:0/2647845449 shutdown_connections 2024-03-18T15:33:52.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.373+0000 7f78c9630640 1 -- 172.21.15.5:0/2647845449 wait complete. 2024-03-18T15:33:52.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.373+0000 7f78c9630640 1 Processor -- start 2024-03-18T15:33:52.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.373+0000 7f78c9630640 1 -- start start 2024-03-18T15:33:52.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.373+0000 7f78c9630640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78c40b1c30 0x7f78c41e3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:52.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.373+0000 7f78c9630640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78c40b65c0 0x7f78c406d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:52.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.373+0000 7f78c9630640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78c4148950 0x7f78c406c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:52.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.373+0000 7f78c9630640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f78c4152830 con 0x7f78c4148950 2024-03-18T15:33:52.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.373+0000 7f78c9630640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f78c41526b0 con 0x7f78c40b1c30 2024-03-18T15:33:52.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.373+0000 7f78c9630640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f78c41529b0 con 0x7f78c40b65c0 2024-03-18T15:33:52.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.373+0000 7f78c37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78c4148950 0x7f78c406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:52.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.373+0000 7f78c37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78c4148950 0x7f78c406c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42936/0 (socket says 172.21.15.5:42936) 2024-03-18T15:33:52.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.373+0000 7f78c37fe640 1 -- 172.21.15.5:0/1457371254 learned_addr learned my addr 172.21.15.5:0/1457371254 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:52.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.373+0000 7f78c27fc640 1 --2- 172.21.15.5:0/1457371254 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78c40b65c0 0x7f78c406d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:52.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.373+0000 7f78c2ffd640 1 --2- 172.21.15.5:0/1457371254 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78c40b1c30 0x7f78c41e3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:52.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.373+0000 7f78c37fe640 1 -- 172.21.15.5:0/1457371254 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78c40b65c0 msgr2=0x7f78c406d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:52.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.373+0000 7f78c37fe640 1 --2- 172.21.15.5:0/1457371254 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78c40b65c0 0x7f78c406d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:52.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.373+0000 7f78c37fe640 1 -- 172.21.15.5:0/1457371254 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78c40b1c30 msgr2=0x7f78c41e3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:52.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.373+0000 7f78c37fe640 1 --2- 172.21.15.5:0/1457371254 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78c40b1c30 0x7f78c41e3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:52.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.373+0000 7f78c37fe640 1 -- 172.21.15.5:0/1457371254 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c4145dc0 con 0x7f78c4148950 2024-03-18T15:33:52.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.373+0000 7f78c37fe640 1 --2- 172.21.15.5:0/1457371254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78c4148950 0x7f78c406c540 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f78bc0ae040 tx=0x7f78bc019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:52.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.373+0000 7f78ab7fe640 1 -- 172.21.15.5:0/1457371254 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78bc0b0040 con 0x7f78c4148950 2024-03-18T15:33:52.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.373+0000 7f78ab7fe640 1 -- 172.21.15.5:0/1457371254 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78bc0b9070 con 0x7f78c4148950 2024-03-18T15:33:52.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.373+0000 7f78ab7fe640 1 -- 172.21.15.5:0/1457371254 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78bc0b4440 con 0x7f78c4148950 2024-03-18T15:33:52.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.373+0000 7f78c9630640 1 -- 172.21.15.5:0/1457371254 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78c406d8f0 con 0x7f78c4148950 2024-03-18T15:33:52.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.373+0000 7f78c9630640 1 -- 172.21.15.5:0/1457371254 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78c4142f90 con 0x7f78c4148950 2024-03-18T15:33:52.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.377+0000 7f78c9630640 1 -- 172.21.15.5:0/1457371254 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78c404f550 con 0x7f78c4148950 2024-03-18T15:33:52.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.377+0000 7f78ab7fe640 1 -- 172.21.15.5:0/1457371254 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f78bc0b45e0 con 0x7f78c4148950 2024-03-18T15:33:52.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.377+0000 7f78ab7fe640 1 --2- 172.21.15.5:0/1457371254 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f78b0067660 0x7f78b0069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:52.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.377+0000 7f78ab7fe640 1 -- 172.21.15.5:0/1457371254 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f78bc11f350 con 0x7f78c4148950 2024-03-18T15:33:52.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.377+0000 7f78c2ffd640 1 --2- 172.21.15.5:0/1457371254 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f78b0067660 0x7f78b0069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:52.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.377+0000 7f78c2ffd640 1 --2- 172.21.15.5:0/1457371254 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f78b0067660 0x7f78b0069b20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f78b4003fd0 tx=0x7f78b40039a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:52.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.381+0000 7f78ab7fe640 1 -- 172.21.15.5:0/1457371254 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f78bc0160d0 con 0x7f78c4148950 2024-03-18T15:33:52.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.537+0000 7f78c9630640 1 -- 172.21.15.5:0/1457371254 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f78c406b940 con 0x7f78c4148950 2024-03-18T15:33:52.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.537+0000 7f78ab7fe640 1 -- 172.21.15.5:0/1457371254 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f78bc0f0060 con 0x7f78c4148950 2024-03-18T15:33:52.538 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-03-18T15:33:52.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.537+0000 7f78c9630640 1 -- 172.21.15.5:0/1457371254 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f78b0067660 msgr2=0x7f78b0069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:52.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.537+0000 7f78c9630640 1 --2- 172.21.15.5:0/1457371254 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f78b0067660 0x7f78b0069b20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f78b4003fd0 tx=0x7f78b40039a0 comp rx=0 tx=0).stop 2024-03-18T15:33:52.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.537+0000 7f78c9630640 1 -- 172.21.15.5:0/1457371254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78c4148950 msgr2=0x7f78c406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:52.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.537+0000 7f78c9630640 1 --2- 172.21.15.5:0/1457371254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78c4148950 0x7f78c406c540 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f78bc0ae040 tx=0x7f78bc019160 comp rx=0 tx=0).stop 2024-03-18T15:33:52.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.537+0000 7f78c9630640 1 -- 172.21.15.5:0/1457371254 shutdown_connections 2024-03-18T15:33:52.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.537+0000 7f78c9630640 1 --2- 172.21.15.5:0/1457371254 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f78b0067660 0x7f78b0069b20 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:52.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.537+0000 7f78c9630640 1 --2- 172.21.15.5:0/1457371254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78c4148950 0x7f78c406c540 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:52.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.537+0000 7f78c9630640 1 --2- 172.21.15.5:0/1457371254 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78c40b65c0 0x7f78c406d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:52.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.537+0000 7f78c9630640 1 --2- 172.21.15.5:0/1457371254 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78c40b1c30 0x7f78c41e3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:52.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.537+0000 7f78c9630640 1 -- 172.21.15.5:0/1457371254 >> 172.21.15.5:0/1457371254 conn(0x7f78c40bacb0 msgr2=0x7f78c40b8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:52.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.537+0000 7f78c9630640 1 -- 172.21.15.5:0/1457371254 shutdown_connections 2024-03-18T15:33:52.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.537+0000 7f78c9630640 1 -- 172.21.15.5:0/1457371254 wait complete. 2024-03-18T15:33:52.555 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-18T15:33:52.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.677+0000 7f94cb7a0640 1 Processor -- start 2024-03-18T15:33:52.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.677+0000 7f94cb7a0640 1 -- start start 2024-03-18T15:33:52.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.677+0000 7f94cb7a0640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f94c413f6a0 0x7f94c41513a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:52.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.677+0000 7f94cb7a0640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94c40b2ab0 con 0x7f94c406d2d0 2024-03-18T15:33:52.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.677+0000 7f94cb7a0640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94c40b0ea0 con 0x7f94c413f260 2024-03-18T15:33:52.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.677+0000 7f94cb7a0640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f94c40728c0 con 0x7f94c413f6a0 2024-03-18T15:33:52.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.677+0000 7f94c9d3d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f94c413f6a0 0x7f94c41513a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:52.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.677+0000 7f94c9d3d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f94c413f6a0 0x7f94c41513a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42952/0 (socket says 172.21.15.5:42952) 2024-03-18T15:33:52.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.677+0000 7f94c9d3d640 1 -- 172.21.15.5:0/3007132762 learned_addr learned my addr 172.21.15.5:0/3007132762 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:52.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.677+0000 7f94c953c640 1 --1- 172.21.15.5:0/3007132762 >> v1:172.21.15.5:6789/0 conn(0x7f94c406d2d0 0x7f94c40b23a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46900/0 (socket says 172.21.15.5:46900) 2024-03-18T15:33:52.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.677+0000 7f94c9d3d640 1 -- 172.21.15.5:0/3007132762 >> v1:172.21.15.80:6789/0 conn(0x7f94c413f260 legacy=0x7f94c414dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:52.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.677+0000 7f94c9d3d640 1 -- 172.21.15.5:0/3007132762 >> v1:172.21.15.5:6789/0 conn(0x7f94c406d2d0 legacy=0x7f94c40b23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:52.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.677+0000 7f94c9d3d640 1 -- 172.21.15.5:0/3007132762 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c406d6b0 con 0x7f94c413f6a0 2024-03-18T15:33:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.677+0000 7f94c9d3d640 1 --2- 172.21.15.5:0/3007132762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94c413f6a0 0x7f94c41513a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f94bc009e50 tx=0x7f94bc01c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9a09bcf582b0414 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.677+0000 7f94b7fff640 1 -- 172.21.15.5:0/3007132762 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94bc0a5690 con 0x7f94c413f6a0 2024-03-18T15:33:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.681+0000 7f94b7fff640 1 -- 172.21.15.5:0/3007132762 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94bc0a5830 con 0x7f94c413f6a0 2024-03-18T15:33:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.681+0000 7f94b7fff640 1 -- 172.21.15.5:0/3007132762 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94bc0a5c00 con 0x7f94c413f6a0 2024-03-18T15:33:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.681+0000 7f94cb7a0640 1 -- 172.21.15.5:0/3007132762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94c413f6a0 msgr2=0x7f94c41513a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.681+0000 7f94cb7a0640 1 --2- 172.21.15.5:0/3007132762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94c413f6a0 0x7f94c41513a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f94bc009e50 tx=0x7f94bc01c830 comp rx=0 tx=0).stop 2024-03-18T15:33:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.681+0000 7f94cb7a0640 1 -- 172.21.15.5:0/3007132762 shutdown_connections 2024-03-18T15:33:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.681+0000 7f94cb7a0640 1 --2- 172.21.15.5:0/3007132762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94c413f6a0 0x7f94c41513a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.681+0000 7f94cb7a0640 1 -- 172.21.15.5:0/3007132762 >> 172.21.15.5:0/3007132762 conn(0x7f94c40b5670 msgr2=0x7f94c40b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.681+0000 7f94cb7a0640 1 -- 172.21.15.5:0/3007132762 shutdown_connections 2024-03-18T15:33:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.681+0000 7f94cb7a0640 1 -- 172.21.15.5:0/3007132762 wait complete. 2024-03-18T15:33:52.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.681+0000 7f94cb7a0640 1 Processor -- start 2024-03-18T15:33:52.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.681+0000 7f94cb7a0640 1 -- start start 2024-03-18T15:33:52.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.681+0000 7f94cb7a0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f94c406d2d0 0x7f94c41dea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:52.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.681+0000 7f94cb7a0640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f94c413f260 0x7f94c4144f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:52.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.681+0000 7f94cb7a0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94c413f6a0 0x7f94c41439d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:52.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.681+0000 7f94cb7a0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f94c40b1c30 con 0x7f94c413f6a0 2024-03-18T15:33:52.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.681+0000 7f94cb7a0640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f94c40b0bc0 con 0x7f94c406d2d0 2024-03-18T15:33:52.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.681+0000 7f94cb7a0640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f94c40b0070 con 0x7f94c413f260 2024-03-18T15:33:52.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.681+0000 7f94c9d3d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94c413f6a0 0x7f94c41439d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:52.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.681+0000 7f94c9d3d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94c413f6a0 0x7f94c41439d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42968/0 (socket says 172.21.15.5:42968) 2024-03-18T15:33:52.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.681+0000 7f94c9d3d640 1 -- 172.21.15.5:0/6527347 learned_addr learned my addr 172.21.15.5:0/6527347 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:52.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.681+0000 7f94c8d3b640 1 --2- 172.21.15.5:0/6527347 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f94c413f260 0x7f94c4144f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:52.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.681+0000 7f94c953c640 1 --2- 172.21.15.5:0/6527347 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f94c406d2d0 0x7f94c41dea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:52.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.681+0000 7f94c9d3d640 1 -- 172.21.15.5:0/6527347 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f94c413f260 msgr2=0x7f94c4144f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:52.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.681+0000 7f94c9d3d640 1 --2- 172.21.15.5:0/6527347 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f94c413f260 0x7f94c4144f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:52.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.681+0000 7f94c9d3d640 1 -- 172.21.15.5:0/6527347 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f94c406d2d0 msgr2=0x7f94c41dea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:52.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.681+0000 7f94c9d3d640 1 --2- 172.21.15.5:0/6527347 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f94c406d2d0 0x7f94c41dea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:52.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.681+0000 7f94c9d3d640 1 -- 172.21.15.5:0/6527347 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c41ddf90 con 0x7f94c413f6a0 2024-03-18T15:33:52.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.681+0000 7f94c8d3b640 1 --2- 172.21.15.5:0/6527347 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f94c413f260 0x7f94c4144f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:33:52.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.681+0000 7f94c953c640 1 --2- 172.21.15.5:0/6527347 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f94c406d2d0 0x7f94c41dea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:33:52.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.681+0000 7f94c9d3d640 1 --2- 172.21.15.5:0/6527347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94c413f6a0 0x7f94c41439d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f94bc0af040 tx=0x7f94bc0195d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:52.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.681+0000 7f94b5ffb640 1 -- 172.21.15.5:0/6527347 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94bc0bc020 con 0x7f94c413f6a0 2024-03-18T15:33:52.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.681+0000 7f94b5ffb640 1 -- 172.21.15.5:0/6527347 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94bc007bb0 con 0x7f94c413f6a0 2024-03-18T15:33:52.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.681+0000 7f94cb7a0640 1 -- 172.21.15.5:0/6527347 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94c41de230 con 0x7f94c413f6a0 2024-03-18T15:33:52.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.681+0000 7f94b5ffb640 1 -- 172.21.15.5:0/6527347 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94bc0a5690 con 0x7f94c413f6a0 2024-03-18T15:33:52.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.681+0000 7f94cb7a0640 1 -- 172.21.15.5:0/6527347 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94c406f7d0 con 0x7f94c413f6a0 2024-03-18T15:33:52.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.685+0000 7f94cb7a0640 1 -- 172.21.15.5:0/6527347 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94c404f550 con 0x7f94c413f6a0 2024-03-18T15:33:52.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.685+0000 7f94b5ffb640 1 -- 172.21.15.5:0/6527347 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f94bc0bf020 con 0x7f94c413f6a0 2024-03-18T15:33:52.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.685+0000 7f94b5ffb640 1 --2- 172.21.15.5:0/6527347 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f94a8067660 0x7f94a8069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:52.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.685+0000 7f94b5ffb640 1 -- 172.21.15.5:0/6527347 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f94bc120d20 con 0x7f94c413f6a0 2024-03-18T15:33:52.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.685+0000 7f94c953c640 1 --2- 172.21.15.5:0/6527347 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f94a8067660 0x7f94a8069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:52.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.685+0000 7f94c953c640 1 --2- 172.21.15.5:0/6527347 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f94a8067660 0x7f94a8069b20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f94c0000f70 tx=0x7f94c000b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:52.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.689+0000 7f94b5ffb640 1 -- 172.21.15.5:0/6527347 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f94bc0c6030 con 0x7f94c413f6a0 2024-03-18T15:33:52.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.849+0000 7f94cb7a0640 1 -- 172.21.15.5:0/6527347 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f94c413d4a0 con 0x7f94c413f6a0 2024-03-18T15:33:52.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.853+0000 7f94b5ffb640 1 -- 172.21.15.5:0/6527347 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f94bc016540 con 0x7f94c413f6a0 2024-03-18T15:33:52.854 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-03-18T15:33:52.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.853+0000 7f94cb7a0640 1 -- 172.21.15.5:0/6527347 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f94a8067660 msgr2=0x7f94a8069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:52.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.853+0000 7f94cb7a0640 1 --2- 172.21.15.5:0/6527347 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f94a8067660 0x7f94a8069b20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f94c0000f70 tx=0x7f94c000b040 comp rx=0 tx=0).stop 2024-03-18T15:33:52.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.853+0000 7f94cb7a0640 1 -- 172.21.15.5:0/6527347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94c413f6a0 msgr2=0x7f94c41439d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:52.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.853+0000 7f94cb7a0640 1 --2- 172.21.15.5:0/6527347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94c413f6a0 0x7f94c41439d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f94bc0af040 tx=0x7f94bc0195d0 comp rx=0 tx=0).stop 2024-03-18T15:33:52.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.853+0000 7f94cb7a0640 1 -- 172.21.15.5:0/6527347 shutdown_connections 2024-03-18T15:33:52.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.853+0000 7f94cb7a0640 1 --2- 172.21.15.5:0/6527347 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f94a8067660 0x7f94a8069b20 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:52.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.853+0000 7f94cb7a0640 1 --2- 172.21.15.5:0/6527347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94c413f6a0 0x7f94c41439d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:52.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.853+0000 7f94cb7a0640 1 --2- 172.21.15.5:0/6527347 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f94c413f260 0x7f94c4144f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:52.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.853+0000 7f94cb7a0640 1 --2- 172.21.15.5:0/6527347 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f94c406d2d0 0x7f94c41dea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:52.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.853+0000 7f94cb7a0640 1 -- 172.21.15.5:0/6527347 >> 172.21.15.5:0/6527347 conn(0x7f94c40b5670 msgr2=0x7f94c40b2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:52.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.853+0000 7f94cb7a0640 1 -- 172.21.15.5:0/6527347 shutdown_connections 2024-03-18T15:33:52.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:52.853+0000 7f94cb7a0640 1 -- 172.21.15.5:0/6527347 wait complete. 2024-03-18T15:33:52.872 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-18T15:33:53.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.001+0000 7fbc8b703640 1 Processor -- start 2024-03-18T15:33:53.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.001+0000 7fbc8b703640 1 -- start start 2024-03-18T15:33:53.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.001+0000 7fbc8b703640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbc84145cc0 0x7fbc84142430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:53.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.001+0000 7fbc8b703640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc840b55f0 con 0x7fbc840b3050 2024-03-18T15:33:53.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.001+0000 7fbc8b703640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc840748b0 con 0x7fbc840744d0 2024-03-18T15:33:53.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.001+0000 7fbc8b703640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fbc8406fa50 con 0x7fbc84145cc0 2024-03-18T15:33:53.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.001+0000 7fbc89ca0640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fbc840b3050 0x7fbc8414fc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46910/0 (socket says 172.21.15.5:46910) 2024-03-18T15:33:53.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.001+0000 7fbc89ca0640 1 -- 172.21.15.5:0/3227778430 learned_addr learned my addr 172.21.15.5:0/3227778430 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:33:53.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.001+0000 7fbc88c9e640 1 --2- 172.21.15.5:0/3227778430 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbc84145cc0 0x7fbc84142430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:53.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.001+0000 7fbc7bfff640 1 -- 172.21.15.5:0/3227778430 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 953681622 0 0) 0x7fbc7c0014a0 con 0x7fbc840b3050 2024-03-18T15:33:53.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.001+0000 7fbc7bfff640 1 -- 172.21.15.5:0/3227778430 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc70003530 con 0x7fbc840b3050 2024-03-18T15:33:53.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.001+0000 7fbc7bfff640 1 -- 172.21.15.5:0/3227778430 <== mon.1 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2505358469 0 0) 0x7fbc840748b0 con 0x7fbc840744d0 2024-03-18T15:33:53.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.001+0000 7fbc7bfff640 1 -- 172.21.15.5:0/3227778430 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc7c0014a0 con 0x7fbc840744d0 2024-03-18T15:33:53.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.001+0000 7fbc7bfff640 1 -- 172.21.15.5:0/3227778430 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1471735574 0 0) 0x7fbc70003530 con 0x7fbc840b3050 2024-03-18T15:33:53.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.001+0000 7fbc7bfff640 1 -- 172.21.15.5:0/3227778430 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbc84145cc0 msgr2=0x7fbc84142430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:53.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.001+0000 7fbc7bfff640 1 --2- 172.21.15.5:0/3227778430 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbc84145cc0 0x7fbc84142430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:53.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.001+0000 7fbc7bfff640 1 -- 172.21.15.5:0/3227778430 >> v1:172.21.15.80:6789/0 conn(0x7fbc840744d0 legacy=0x7fbc841455b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:53.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.001+0000 7fbc7bfff640 1 -- 172.21.15.5:0/3227778430 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc840711f0 con 0x7fbc840b3050 2024-03-18T15:33:53.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.001+0000 7fbc7bfff640 1 -- 172.21.15.5:0/3227778430 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3837549003 0 0) 0x7fbc7c001bd0 con 0x7fbc840b3050 2024-03-18T15:33:53.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.001+0000 7fbc7bfff640 1 -- 172.21.15.5:0/3227778430 >> v1:172.21.15.5:6789/0 conn(0x7fbc840b3050 legacy=0x7fbc8414fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:53.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.001+0000 7fbc7bfff640 1 --2- 172.21.15.5:0/3227778430 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbc84145cc0 0x7fbc70003c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:53.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.001+0000 7fbc7bfff640 1 --2- 172.21.15.5:0/3227778430 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc700041c0 0x7fbc70004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:53.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.001+0000 7fbc7bfff640 1 --2- 172.21.15.5:0/3227778430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc70004d50 0x7fbc700051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:53.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.001+0000 7fbc7bfff640 1 -- 172.21.15.5:0/3227778430 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbc8406fa50 con 0x7fbc70004d50 2024-03-18T15:33:53.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.001+0000 7fbc7bfff640 1 -- 172.21.15.5:0/3227778430 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fbc7000bc10 con 0x7fbc700041c0 2024-03-18T15:33:53.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.001+0000 7fbc7bfff640 1 -- 172.21.15.5:0/3227778430 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fbc7000be10 con 0x7fbc84145cc0 2024-03-18T15:33:53.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.001+0000 7fbc89ca0640 1 --2- 172.21.15.5:0/3227778430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc70004d50 0x7fbc700051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:53.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.001+0000 7fbc8949f640 1 --2- 172.21.15.5:0/3227778430 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbc84145cc0 0x7fbc70003c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:53.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.001+0000 7fbc88c9e640 1 --2- 172.21.15.5:0/3227778430 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc700041c0 0x7fbc70004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:53.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.001+0000 7fbc89ca0640 1 -- 172.21.15.5:0/3227778430 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbc84145cc0 msgr2=0x7fbc70003c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:53.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.001+0000 7fbc89ca0640 1 --2- 172.21.15.5:0/3227778430 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbc84145cc0 0x7fbc70003c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:53.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.001+0000 7fbc89ca0640 1 -- 172.21.15.5:0/3227778430 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc700041c0 msgr2=0x7fbc70004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:53.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.001+0000 7fbc89ca0640 1 --2- 172.21.15.5:0/3227778430 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc700041c0 0x7fbc70004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:53.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.001+0000 7fbc89ca0640 1 -- 172.21.15.5:0/3227778430 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc70003530 con 0x7fbc70004d50 2024-03-18T15:33:53.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.005+0000 7fbc89ca0640 1 --2- 172.21.15.5:0/3227778430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc70004d50 0x7fbc700051d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c00a9b0 tx=0x7fbc7c01d300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39fbc4639846afaf server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:53.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.005+0000 7fbc7bfff640 1 -- 172.21.15.5:0/3227778430 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c00a600 con 0x7fbc70004d50 2024-03-18T15:33:53.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.005+0000 7fbc7bfff640 1 -- 172.21.15.5:0/3227778430 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc7c0ad630 con 0x7fbc70004d50 2024-03-18T15:33:53.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.005+0000 7fbc7bfff640 1 -- 172.21.15.5:0/3227778430 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c0ada20 con 0x7fbc70004d50 2024-03-18T15:33:53.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.005+0000 7fbc8b703640 1 -- 172.21.15.5:0/3227778430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc70004d50 msgr2=0x7fbc700051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:53.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.005+0000 7fbc8b703640 1 --2- 172.21.15.5:0/3227778430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc70004d50 0x7fbc700051d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c00a9b0 tx=0x7fbc7c01d300 comp rx=0 tx=0).stop 2024-03-18T15:33:53.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.005+0000 7fbc89ca0640 1 -- 172.21.15.5:0/3227778430 reap_dead start 2024-03-18T15:33:53.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.005+0000 7fbc8b703640 1 -- 172.21.15.5:0/3227778430 shutdown_connections 2024-03-18T15:33:53.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.005+0000 7fbc8b703640 1 -- 172.21.15.5:0/3227778430 >> 172.21.15.5:0/3227778430 conn(0x7fbc8406bad0 msgr2=0x7fbc840b9190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:53.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.005+0000 7fbc8b703640 1 -- 172.21.15.5:0/3227778430 shutdown_connections 2024-03-18T15:33:53.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.005+0000 7fbc8b703640 1 -- 172.21.15.5:0/3227778430 wait complete. 2024-03-18T15:33:53.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.005+0000 7fbc8b703640 1 Processor -- start 2024-03-18T15:33:53.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.005+0000 7fbc8b703640 1 -- start start 2024-03-18T15:33:53.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.009+0000 7fbc8b703640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbc70004d50 0x7fbc841de9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:53.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.009+0000 7fbc8b703640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc8413dc20 0x7fbc8413e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:53.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.009+0000 7fbc8b703640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc84147110 0x7fbc84147590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:53.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.009+0000 7fbc8b703640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbc84070f10 con 0x7fbc84147110 2024-03-18T15:33:53.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.009+0000 7fbc8b703640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fbc84070010 con 0x7fbc8413dc20 2024-03-18T15:33:53.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.009+0000 7fbc8b703640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fbc84142430 con 0x7fbc70004d50 2024-03-18T15:33:53.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.009+0000 7fbc89ca0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc84147110 0x7fbc84147590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:53.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.009+0000 7fbc89ca0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc84147110 0x7fbc84147590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42982/0 (socket says 172.21.15.5:42982) 2024-03-18T15:33:53.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.009+0000 7fbc89ca0640 1 -- 172.21.15.5:0/2807861804 learned_addr learned my addr 172.21.15.5:0/2807861804 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:53.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.009+0000 7fbc88c9e640 1 --2- 172.21.15.5:0/2807861804 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc8413dc20 0x7fbc8413e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:53.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.009+0000 7fbc8949f640 1 --2- 172.21.15.5:0/2807861804 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbc70004d50 0x7fbc841de9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:53.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.009+0000 7fbc89ca0640 1 -- 172.21.15.5:0/2807861804 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbc70004d50 msgr2=0x7fbc841de9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:53.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.009+0000 7fbc89ca0640 1 --2- 172.21.15.5:0/2807861804 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbc70004d50 0x7fbc841de9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:53.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.009+0000 7fbc89ca0640 1 -- 172.21.15.5:0/2807861804 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc8413dc20 msgr2=0x7fbc8413e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:53.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.009+0000 7fbc89ca0640 1 --2- 172.21.15.5:0/2807861804 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc8413dc20 0x7fbc8413e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:53.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.009+0000 7fbc89ca0640 1 -- 172.21.15.5:0/2807861804 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc8413e5a0 con 0x7fbc84147110 2024-03-18T15:33:53.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.009+0000 7fbc89ca0640 1 --2- 172.21.15.5:0/2807861804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc84147110 0x7fbc84147590 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c01d130 tx=0x7fbc70005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:53.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.009+0000 7fbc79ffb640 1 -- 172.21.15.5:0/2807861804 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c00a5e0 con 0x7fbc84147110 2024-03-18T15:33:53.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.009+0000 7fbc79ffb640 1 -- 172.21.15.5:0/2807861804 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc7c0ad430 con 0x7fbc84147110 2024-03-18T15:33:53.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.009+0000 7fbc79ffb640 1 -- 172.21.15.5:0/2807861804 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c0ad7e0 con 0x7fbc84147110 2024-03-18T15:33:53.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.009+0000 7fbc8b703640 1 -- 172.21.15.5:0/2807861804 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc841df290 con 0x7fbc84147110 2024-03-18T15:33:53.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.009+0000 7fbc8b703640 1 -- 172.21.15.5:0/2807861804 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc84146570 con 0x7fbc84147110 2024-03-18T15:33:53.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.009+0000 7fbc8b703640 1 -- 172.21.15.5:0/2807861804 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc8404f550 con 0x7fbc84147110 2024-03-18T15:33:53.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.009+0000 7fbc79ffb640 1 -- 172.21.15.5:0/2807861804 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fbc7c0bd060 con 0x7fbc84147110 2024-03-18T15:33:53.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.009+0000 7fbc79ffb640 1 --2- 172.21.15.5:0/2807861804 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fbc68067660 0x7fbc68069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:53.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.009+0000 7fbc79ffb640 1 -- 172.21.15.5:0/2807861804 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fbc7c11f5b0 con 0x7fbc84147110 2024-03-18T15:33:53.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.009+0000 7fbc8949f640 1 --2- 172.21.15.5:0/2807861804 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fbc68067660 0x7fbc68069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:53.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.009+0000 7fbc8949f640 1 --2- 172.21.15.5:0/2807861804 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fbc68067660 0x7fbc68069b20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbc80000c10 tx=0x7fbc80001490 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:53.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.013+0000 7fbc79ffb640 1 -- 172.21.15.5:0/2807861804 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fbc840b55f0 con 0x7fbc84147110 2024-03-18T15:33:53.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.173+0000 7fbc8b703640 1 -- 172.21.15.5:0/2807861804 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbc84070950 con 0x7fbc84147110 2024-03-18T15:33:53.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.173+0000 7fbc79ffb640 1 -- 172.21.15.5:0/2807861804 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7fbc7c0f0230 con 0x7fbc84147110 2024-03-18T15:33:53.174 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-03-18T15:33:53.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.173+0000 7fbc8b703640 1 -- 172.21.15.5:0/2807861804 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fbc68067660 msgr2=0x7fbc68069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:53.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.173+0000 7fbc8b703640 1 --2- 172.21.15.5:0/2807861804 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fbc68067660 0x7fbc68069b20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbc80000c10 tx=0x7fbc80001490 comp rx=0 tx=0).stop 2024-03-18T15:33:53.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.173+0000 7fbc8b703640 1 -- 172.21.15.5:0/2807861804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc84147110 msgr2=0x7fbc84147590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:53.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.173+0000 7fbc8b703640 1 --2- 172.21.15.5:0/2807861804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc84147110 0x7fbc84147590 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c01d130 tx=0x7fbc70005840 comp rx=0 tx=0).stop 2024-03-18T15:33:53.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.173+0000 7fbc8b703640 1 -- 172.21.15.5:0/2807861804 shutdown_connections 2024-03-18T15:33:53.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.173+0000 7fbc8b703640 1 --2- 172.21.15.5:0/2807861804 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fbc68067660 0x7fbc68069b20 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:53.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.173+0000 7fbc8b703640 1 --2- 172.21.15.5:0/2807861804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc84147110 0x7fbc84147590 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:53.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.173+0000 7fbc8b703640 1 --2- 172.21.15.5:0/2807861804 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc8413dc20 0x7fbc8413e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:53.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.173+0000 7fbc8b703640 1 --2- 172.21.15.5:0/2807861804 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbc70004d50 0x7fbc841de9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:53.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.173+0000 7fbc8b703640 1 -- 172.21.15.5:0/2807861804 >> 172.21.15.5:0/2807861804 conn(0x7fbc8406bad0 msgr2=0x7fbc840690d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:53.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.173+0000 7fbc8b703640 1 -- 172.21.15.5:0/2807861804 shutdown_connections 2024-03-18T15:33:53.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.173+0000 7fbc8b703640 1 -- 172.21.15.5:0/2807861804 wait complete. 2024-03-18T15:33:53.192 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:33:53.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.297+0000 7fdfb63ac640 1 Processor -- start 2024-03-18T15:33:53.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.297+0000 7fdfb63ac640 1 -- start start 2024-03-18T15:33:53.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.301+0000 7fdfb63ac640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fdfb014b030 0x7fdfb0140ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:53.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.301+0000 7fdfb63ac640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fdfb0141430 0x7fdfb0151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:53.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.301+0000 7fdfb63ac640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfb00b9d50 con 0x7fdfb00b5710 2024-03-18T15:33:53.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.301+0000 7fdfb63ac640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fdfb00b4320 con 0x7fdfb014b030 2024-03-18T15:33:53.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.301+0000 7fdfb63ac640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fdfb00b4570 con 0x7fdfb0141430 2024-03-18T15:33:53.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.301+0000 7fdfa7fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fdfb014b030 0x7fdfb0140ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:53.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.301+0000 7fdfa7fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fdfb014b030 0x7fdfb0140ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42986/0 (socket says 172.21.15.5:42986) 2024-03-18T15:33:53.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.301+0000 7fdfa7fff640 1 -- 172.21.15.5:0/1640893238 learned_addr learned my addr 172.21.15.5:0/1640893238 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:53.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.301+0000 7fdfb4949640 1 --2- 172.21.15.5:0/1640893238 >> v2:172.21.15.80:3300/0 conn(0x7fdfb0141430 0x7fdfb0151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:53.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.301+0000 7fdfa7fff640 1 -- 172.21.15.5:0/1640893238 >> v2:172.21.15.80:3300/0 conn(0x7fdfb0141430 msgr2=0x7fdfb0151cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:53.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.301+0000 7fdfa7fff640 1 --2- 172.21.15.5:0/1640893238 >> v2:172.21.15.80:3300/0 conn(0x7fdfb0141430 0x7fdfb0151cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:53.303 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.301+0000 7fdfa7fff640 1 -- 172.21.15.5:0/1640893238 >> v1:172.21.15.80:6789/0 conn(0x7fdfb00b5710 legacy=0x7fdfb014a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:53.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.301+0000 7fdfa7fff640 1 -- 172.21.15.5:0/1640893238 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfb00b4c30 con 0x7fdfb014b030 2024-03-18T15:33:53.304 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.301+0000 7fdfa7fff640 1 --2- 172.21.15.5:0/1640893238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfb014b030 0x7fdfb0140ef0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdfb00b6e10 tx=0x7fdfa001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=560eff8b8d1b9fb2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:53.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.301+0000 7fdfaf7fe640 1 -- 172.21.15.5:0/1640893238 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa00b0070 con 0x7fdfb014b030 2024-03-18T15:33:53.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.301+0000 7fdfaf7fe640 1 -- 172.21.15.5:0/1640893238 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfa00a4e50 con 0x7fdfb014b030 2024-03-18T15:33:53.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.301+0000 7fdfaf7fe640 1 -- 172.21.15.5:0/1640893238 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa00ab820 con 0x7fdfb014b030 2024-03-18T15:33:53.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.301+0000 7fdfb63ac640 1 -- 172.21.15.5:0/1640893238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfb014b030 msgr2=0x7fdfb0140ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:53.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.301+0000 7fdfb63ac640 1 --2- 172.21.15.5:0/1640893238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfb014b030 0x7fdfb0140ef0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fdfb00b6e10 tx=0x7fdfa001c3c0 comp rx=0 tx=0).stop 2024-03-18T15:33:53.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.305+0000 7fdfb63ac640 1 -- 172.21.15.5:0/1640893238 shutdown_connections 2024-03-18T15:33:53.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.305+0000 7fdfb63ac640 1 --2- 172.21.15.5:0/1640893238 >> v2:172.21.15.80:3300/0 conn(0x7fdfb0141430 0x7fdfb0151cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:53.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.305+0000 7fdfb63ac640 1 --2- 172.21.15.5:0/1640893238 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfb014b030 0x7fdfb0140ef0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:53.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.305+0000 7fdfb63ac640 1 -- 172.21.15.5:0/1640893238 >> 172.21.15.5:0/1640893238 conn(0x7fdfb00673c0 msgr2=0x7fdfb00677d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:53.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.305+0000 7fdfb63ac640 1 -- 172.21.15.5:0/1640893238 shutdown_connections 2024-03-18T15:33:53.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.305+0000 7fdfb63ac640 1 -- 172.21.15.5:0/1640893238 wait complete. 2024-03-18T15:33:53.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.305+0000 7fdfb63ac640 1 Processor -- start 2024-03-18T15:33:53.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.305+0000 7fdfb63ac640 1 -- start start 2024-03-18T15:33:53.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.305+0000 7fdfb63ac640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfb00b5710 0x7fdfb013fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:53.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.305+0000 7fdfb63ac640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdfb0141430 0x7fdfb00bcd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:53.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.305+0000 7fdfb63ac640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdfb014b030 0x7fdfb013dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:53.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.305+0000 7fdfb63ac640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb01419c0 con 0x7fdfb00b5710 2024-03-18T15:33:53.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.305+0000 7fdfb63ac640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb0141840 con 0x7fdfb0141430 2024-03-18T15:33:53.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.305+0000 7fdfb63ac640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fdfb0154a10 con 0x7fdfb014b030 2024-03-18T15:33:53.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.305+0000 7fdfaffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfb00b5710 0x7fdfb013fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:53.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.305+0000 7fdfaffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfb00b5710 0x7fdfb013fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42990/0 (socket says 172.21.15.5:42990) 2024-03-18T15:33:53.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.305+0000 7fdfaffff640 1 -- 172.21.15.5:0/3119352097 learned_addr learned my addr 172.21.15.5:0/3119352097 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:53.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.305+0000 7fdfaffff640 1 -- 172.21.15.5:0/3119352097 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdfb014b030 msgr2=0x7fdfb013dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:53.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.305+0000 7fdfaffff640 1 --2- 172.21.15.5:0/3119352097 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdfb014b030 0x7fdfb013dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:53.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.305+0000 7fdfaffff640 1 -- 172.21.15.5:0/3119352097 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdfb0141430 msgr2=0x7fdfb00bcd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:53.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.305+0000 7fdfa7fff640 1 --2- 172.21.15.5:0/3119352097 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdfb0141430 0x7fdfb00bcd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:53.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.305+0000 7fdfaffff640 1 --2- 172.21.15.5:0/3119352097 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdfb0141430 0x7fdfb00bcd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:53.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.305+0000 7fdfaffff640 1 -- 172.21.15.5:0/3119352097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfb013c370 con 0x7fdfb00b5710 2024-03-18T15:33:53.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.305+0000 7fdfaffff640 1 --2- 172.21.15.5:0/3119352097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfb00b5710 0x7fdfb013fd70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdf9800c450 tx=0x7fdf9800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:53.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.305+0000 7fdfad7fa640 1 -- 172.21.15.5:0/3119352097 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf98012830 con 0x7fdfb00b5710 2024-03-18T15:33:53.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.305+0000 7fdfad7fa640 1 -- 172.21.15.5:0/3119352097 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf98017070 con 0x7fdfb00b5710 2024-03-18T15:33:53.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.305+0000 7fdfad7fa640 1 -- 172.21.15.5:0/3119352097 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf98012c30 con 0x7fdfb00b5710 2024-03-18T15:33:53.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.305+0000 7fdfb63ac640 1 -- 172.21.15.5:0/3119352097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfb0140390 con 0x7fdfb00b5710 2024-03-18T15:33:53.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.305+0000 7fdfb63ac640 1 -- 172.21.15.5:0/3119352097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfb013f8e0 con 0x7fdfb00b5710 2024-03-18T15:33:53.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.305+0000 7fdfb63ac640 1 -- 172.21.15.5:0/3119352097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfb004f550 con 0x7fdfb00b5710 2024-03-18T15:33:53.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.305+0000 7fdfad7fa640 1 -- 172.21.15.5:0/3119352097 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fdf9801c460 con 0x7fdfb00b5710 2024-03-18T15:33:53.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.309+0000 7fdfad7fa640 1 --2- 172.21.15.5:0/3119352097 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fdf90067660 0x7fdf90069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:53.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.309+0000 7fdfad7fa640 1 -- 172.21.15.5:0/3119352097 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fdf98085600 con 0x7fdfb00b5710 2024-03-18T15:33:53.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.309+0000 7fdfa7fff640 1 --2- 172.21.15.5:0/3119352097 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fdf90067660 0x7fdf90069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:53.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.309+0000 7fdfa7fff640 1 --2- 172.21.15.5:0/3119352097 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fdf90067660 0x7fdf90069b20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdfa00ad000 tx=0x7fdfa00a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:53.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.313+0000 7fdfad7fa640 1 -- 172.21.15.5:0/3119352097 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fdf98056310 con 0x7fdfb00b5710 2024-03-18T15:33:53.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.497+0000 7fdfb63ac640 1 -- 172.21.15.5:0/3119352097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdfb00b8c10 con 0x7fdfb00b5710 2024-03-18T15:33:53.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.501+0000 7fdfad7fa640 1 -- 172.21.15.5:0/3119352097 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+3521 (secure 0 0 0) 0x7fdf98055c10 con 0x7fdfb00b5710 2024-03-18T15:33:53.502 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:33:53.502 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/629589896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":629589896},{"type":"v1","addr":"172.21.15.5:6835","nonce":629589896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-18T15:33:42.338926+0000","modified":"2024-03-18T15:33:45.376646+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6833/1236603350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":1236603350},{"type":"v1","addr":"172.21.15.80:6833","nonce":1236603350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4272,"qdb_cluster":[4272]},"id":1}]} 2024-03-18T15:33:53.502 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 8 2024-03-18T15:33:53.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.501+0000 7fdfb63ac640 1 -- 172.21.15.5:0/3119352097 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fdf90067660 msgr2=0x7fdf90069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:53.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.501+0000 7fdfb63ac640 1 --2- 172.21.15.5:0/3119352097 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fdf90067660 0x7fdf90069b20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdfa00ad000 tx=0x7fdfa00a6000 comp rx=0 tx=0).stop 2024-03-18T15:33:53.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.501+0000 7fdfb63ac640 1 -- 172.21.15.5:0/3119352097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfb00b5710 msgr2=0x7fdfb013fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:53.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.501+0000 7fdfb63ac640 1 --2- 172.21.15.5:0/3119352097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfb00b5710 0x7fdfb013fd70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdf9800c450 tx=0x7fdf9800c910 comp rx=0 tx=0).stop 2024-03-18T15:33:53.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.501+0000 7fdfb63ac640 1 -- 172.21.15.5:0/3119352097 shutdown_connections 2024-03-18T15:33:53.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.501+0000 7fdfb63ac640 1 --2- 172.21.15.5:0/3119352097 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fdf90067660 0x7fdf90069b20 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:53.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.501+0000 7fdfb63ac640 1 --2- 172.21.15.5:0/3119352097 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdfb014b030 0x7fdfb013dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:53.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.501+0000 7fdfb63ac640 1 --2- 172.21.15.5:0/3119352097 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdfb0141430 0x7fdfb00bcd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:53.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.501+0000 7fdfb63ac640 1 --2- 172.21.15.5:0/3119352097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdfb00b5710 0x7fdfb013fd70 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:53.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.501+0000 7fdfb63ac640 1 -- 172.21.15.5:0/3119352097 >> 172.21.15.5:0/3119352097 conn(0x7fdfb00673c0 msgr2=0x7fdfb00b7de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:53.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.501+0000 7fdfb63ac640 1 -- 172.21.15.5:0/3119352097 shutdown_connections 2024-03-18T15:33:53.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.501+0000 7fdfb63ac640 1 -- 172.21.15.5:0/3119352097 wait complete. 2024-03-18T15:33:53.520 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:33:53.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.641+0000 7fe048549640 1 Processor -- start 2024-03-18T15:33:53.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.641+0000 7fe048549640 1 -- start start 2024-03-18T15:33:53.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.641+0000 7fe048549640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe04006d2d0 0x7fe040141650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:53.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.641+0000 7fe048549640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0400ae0f0 0x7fe0400b4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:53.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.641+0000 7fe048549640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe04006d6b0 con 0x7fe0400ae9c0 2024-03-18T15:33:53.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.641+0000 7fe048549640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fe0400724e0 con 0x7fe04006d2d0 2024-03-18T15:33:53.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.641+0000 7fe048549640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fe040072730 con 0x7fe0400ae0f0 2024-03-18T15:33:53.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.641+0000 7fe0462e5640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe04006d2d0 0x7fe040141650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:53.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.641+0000 7fe0462e5640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe04006d2d0 0x7fe040141650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42998/0 (socket says 172.21.15.5:42998) 2024-03-18T15:33:53.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.641+0000 7fe0462e5640 1 -- 172.21.15.5:0/3249643506 learned_addr learned my addr 172.21.15.5:0/3249643506 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:53.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.641+0000 7fe045ae4640 1 --2- 172.21.15.5:0/3249643506 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0400ae0f0 0x7fe0400b4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:53.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.641+0000 7fe0462e5640 1 -- 172.21.15.5:0/3249643506 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0400ae0f0 msgr2=0x7fe0400b4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:53.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.641+0000 7fe0462e5640 1 --2- 172.21.15.5:0/3249643506 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0400ae0f0 0x7fe0400b4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:53.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.641+0000 7fe0462e5640 1 -- 172.21.15.5:0/3249643506 >> v1:172.21.15.80:6789/0 conn(0x7fe0400ae9c0 legacy=0x7fe04014aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:53.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.641+0000 7fe0462e5640 1 -- 172.21.15.5:0/3249643506 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe040072a10 con 0x7fe04006d2d0 2024-03-18T15:33:53.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.641+0000 7fe0462e5640 1 --2- 172.21.15.5:0/3249643506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe04006d2d0 0x7fe040141650 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe030009e50 tx=0x7fe03001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e51263cc2065fa13 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:53.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.641+0000 7fe0452e3640 1 -- 172.21.15.5:0/3249643506 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0300a55e0 con 0x7fe04006d2d0 2024-03-18T15:33:53.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.641+0000 7fe0452e3640 1 -- 172.21.15.5:0/3249643506 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0300a5780 con 0x7fe04006d2d0 2024-03-18T15:33:53.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.641+0000 7fe0452e3640 1 -- 172.21.15.5:0/3249643506 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0300a5b50 con 0x7fe04006d2d0 2024-03-18T15:33:53.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.645+0000 7fe048549640 1 -- 172.21.15.5:0/3249643506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe04006d2d0 msgr2=0x7fe040141650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:53.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.645+0000 7fe048549640 1 --2- 172.21.15.5:0/3249643506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe04006d2d0 0x7fe040141650 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe030009e50 tx=0x7fe03001c960 comp rx=0 tx=0).stop 2024-03-18T15:33:53.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.645+0000 7fe048549640 1 -- 172.21.15.5:0/3249643506 shutdown_connections 2024-03-18T15:33:53.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.645+0000 7fe048549640 1 --2- 172.21.15.5:0/3249643506 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe0400ae0f0 0x7fe0400b4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:53.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.645+0000 7fe048549640 1 --2- 172.21.15.5:0/3249643506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe04006d2d0 0x7fe040141650 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:53.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.645+0000 7fe048549640 1 -- 172.21.15.5:0/3249643506 >> 172.21.15.5:0/3249643506 conn(0x7fe0400b5670 msgr2=0x7fe0400b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:53.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.645+0000 7fe048549640 1 -- 172.21.15.5:0/3249643506 shutdown_connections 2024-03-18T15:33:53.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.645+0000 7fe048549640 1 -- 172.21.15.5:0/3249643506 wait complete. 2024-03-18T15:33:53.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.645+0000 7fe048549640 1 Processor -- start 2024-03-18T15:33:53.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.645+0000 7fe048549640 1 -- start start 2024-03-18T15:33:53.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.645+0000 7fe048549640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe04006d2d0 0x7fe04013d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:53.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.645+0000 7fe048549640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0400ae0f0 0x7fe040149530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:53.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.645+0000 7fe048549640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0400ae9c0 0x7fe0401e3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:53.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.645+0000 7fe048549640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe040155030 con 0x7fe0400ae9c0 2024-03-18T15:33:53.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.645+0000 7fe048549640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe040141da0 con 0x7fe0400ae0f0 2024-03-18T15:33:53.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.645+0000 7fe048549640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fe0401551b0 con 0x7fe04006d2d0 2024-03-18T15:33:53.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.645+0000 7fe046ae6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0400ae9c0 0x7fe0401e3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:53.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.645+0000 7fe046ae6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0400ae9c0 0x7fe0401e3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43002/0 (socket says 172.21.15.5:43002) 2024-03-18T15:33:53.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.645+0000 7fe046ae6640 1 -- 172.21.15.5:0/2273044553 learned_addr learned my addr 172.21.15.5:0/2273044553 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:53.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.645+0000 7fe0462e5640 1 --2- 172.21.15.5:0/2273044553 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe04006d2d0 0x7fe04013d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:53.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.645+0000 7fe045ae4640 1 --2- 172.21.15.5:0/2273044553 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0400ae0f0 0x7fe040149530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:53.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.645+0000 7fe046ae6640 1 -- 172.21.15.5:0/2273044553 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe04006d2d0 msgr2=0x7fe04013d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:53.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.645+0000 7fe046ae6640 1 --2- 172.21.15.5:0/2273044553 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe04006d2d0 0x7fe04013d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:53.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.645+0000 7fe046ae6640 1 -- 172.21.15.5:0/2273044553 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0400ae0f0 msgr2=0x7fe040149530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:53.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.645+0000 7fe046ae6640 1 --2- 172.21.15.5:0/2273044553 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0400ae0f0 0x7fe040149530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:53.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.645+0000 7fe046ae6640 1 -- 172.21.15.5:0/2273044553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0400b8870 con 0x7fe0400ae9c0 2024-03-18T15:33:53.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.645+0000 7fe0462e5640 1 --2- 172.21.15.5:0/2273044553 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe04006d2d0 0x7fe04013d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:33:53.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.645+0000 7fe046ae6640 1 --2- 172.21.15.5:0/2273044553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0400ae9c0 0x7fe0401e3a80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe03800b410 tx=0x7fe03800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:53.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.645+0000 7fe03effd640 1 -- 172.21.15.5:0/2273044553 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe038015020 con 0x7fe0400ae9c0 2024-03-18T15:33:53.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.645+0000 7fe03effd640 1 -- 172.21.15.5:0/2273044553 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe038011830 con 0x7fe0400ae9c0 2024-03-18T15:33:53.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.645+0000 7fe03effd640 1 -- 172.21.15.5:0/2273044553 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe038011c60 con 0x7fe0400ae9c0 2024-03-18T15:33:53.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.645+0000 7fe048549640 1 -- 172.21.15.5:0/2273044553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0401489e0 con 0x7fe0400ae9c0 2024-03-18T15:33:53.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.645+0000 7fe048549640 1 -- 172.21.15.5:0/2273044553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0400b8590 con 0x7fe0400ae9c0 2024-03-18T15:33:53.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.649+0000 7fe048549640 1 -- 172.21.15.5:0/2273044553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe04004f550 con 0x7fe0400ae9c0 2024-03-18T15:33:53.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.649+0000 7fe03effd640 1 -- 172.21.15.5:0/2273044553 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fe038018020 con 0x7fe0400ae9c0 2024-03-18T15:33:53.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.649+0000 7fe03effd640 1 --2- 172.21.15.5:0/2273044553 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe02c067660 0x7fe02c069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:53.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.649+0000 7fe0462e5640 1 --2- 172.21.15.5:0/2273044553 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe02c067660 0x7fe02c069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:53.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.649+0000 7fe03effd640 1 -- 172.21.15.5:0/2273044553 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe038085300 con 0x7fe0400ae9c0 2024-03-18T15:33:53.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.649+0000 7fe0462e5640 1 --2- 172.21.15.5:0/2273044553 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe02c067660 0x7fe02c069b20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe0300ae000 tx=0x7fe0300a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:53.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.653+0000 7fe03effd640 1 -- 172.21.15.5:0/2273044553 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fe03801f030 con 0x7fe0400ae9c0 2024-03-18T15:33:53.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.813+0000 7fe048549640 1 -- 172.21.15.5:0/2273044553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe040144f70 con 0x7fe0400ae9c0 2024-03-18T15:33:53.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.817+0000 7fe03effd640 1 -- 172.21.15.5:0/2273044553 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16449 (secure 0 0 0) 0x7fe0380172c0 con 0x7fe0400ae9c0 2024-03-18T15:33:53.819 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:33:53.819 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":21,"fsid":"ef54fe03-5aa7-4eb2-9b1e-66c9cc345b94","created":"2024-03-18T15:33:18.086614+0000","modified":"2024-03-18T15:33:43.984050+0000","last_up_change":"2024-03-18T15:33:29.229911+0000","last_in_change":"2024-03-18T15:33:20.323840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:28.963186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T15:33:40.533646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T15:33:41.323519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df1c754b-af59-4d16-8919-fbb6386b3c48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6804","nonce":976264754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6808","nonce":976264754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6815","nonce":976264754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6811","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6813","nonce":976264754}]},"public_addr":"172.21.15.5:6804/976264754","cluster_addr":"172.21.15.5:6808/976264754","heartbeat_back_addr":"172.21.15.5:6815/976264754","heartbeat_front_addr":"172.21.15.5:6813/976264754","state":["exists","up"]},{"osd":1,"uuid":"b9a08b5e-8af4-4144-bdba-d69ed1ac627a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6801","nonce":3901761126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6803","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6805","nonce":3901761126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6812","nonce":3901761126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6807","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6809","nonce":3901761126}]},"public_addr":"172.21.15.5:6801/3901761126","cluster_addr":"172.21.15.5:6805/3901761126","heartbeat_back_addr":"172.21.15.5:6812/3901761126","heartbeat_front_addr":"172.21.15.5:6809/3901761126","state":["exists","up"]},{"osd":2,"uuid":"c3e8bb76-e382-439d-b380-62ef92f1255a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6817","nonce":502968448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6819","nonce":502968448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6823","nonce":502968448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6821","nonce":502968448}]},"public_addr":"172.21.15.5:6817/502968448","cluster_addr":"172.21.15.5:6819/502968448","heartbeat_back_addr":"172.21.15.5:6823/502968448","heartbeat_front_addr":"172.21.15.5:6821/502968448","state":["exists","up"]},{"osd":3,"uuid":"e8c94625-f2cd-4eba-b53d-2e7356e1c02a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6825","nonce":2425932735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6827","nonce":2425932735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6831","nonce":2425932735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6829","nonce":2425932735}]},"public_addr":"172.21.15.5:6825/2425932735","cluster_addr":"172.21.15.5:6827/2425932735","heartbeat_back_addr":"172.21.15.5:6831/2425932735","heartbeat_front_addr":"172.21.15.5:6829/2425932735","state":["exists","up"]},{"osd":4,"uuid":"0099dd52-6265-4d32-82e4-3ef16e73add0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6817","nonce":976514285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6819","nonce":976514285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6823","nonce":976514285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6821","nonce":976514285}]},"public_addr":"172.21.15.80:6817/976514285","cluster_addr":"172.21.15.80:6819/976514285","heartbeat_back_addr":"172.21.15.80:6823/976514285","heartbeat_front_addr":"172.21.15.80:6821/976514285","state":["exists","up"]},{"osd":5,"uuid":"ed10c84f-8c39-41cc-9b64-7c482a94a48e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6825","nonce":627153447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6827","nonce":627153447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6831","nonce":627153447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6829","nonce":627153447}]},"public_addr":"172.21.15.80:6825/627153447","cluster_addr":"172.21.15.80:6827/627153447","heartbeat_back_addr":"172.21.15.80:6831/627153447","heartbeat_front_addr":"172.21.15.80:6829/627153447","state":["exists","up"]},{"osd":6,"uuid":"c7a3538a-7048-4fba-8781-b440f03c25db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6803","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6805","nonce":488342527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6807","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6809","nonce":488342527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6815","nonce":488342527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6811","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6813","nonce":488342527}]},"public_addr":"172.21.15.80:6805/488342527","cluster_addr":"172.21.15.80:6809/488342527","heartbeat_back_addr":"172.21.15.80:6815/488342527","heartbeat_front_addr":"172.21.15.80:6813/488342527","state":["exists","up"]},{"osd":7,"uuid":"f9418104-3a97-4110-bc3a-1f8151d0f757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6801","nonce":982831666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6804","nonce":982831666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6812","nonce":982831666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6808","nonce":982831666}]},"public_addr":"172.21.15.80:6801/982831666","cluster_addr":"172.21.15.80:6804/982831666","heartbeat_back_addr":"172.21.15.80:6812/982831666","heartbeat_front_addr":"172.21.15.80:6808/982831666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.970828+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.858116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.987556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.864603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.756293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.591502+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:33:53.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.821+0000 7fe048549640 1 -- 172.21.15.5:0/2273044553 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe02c067660 msgr2=0x7fe02c069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:53.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.821+0000 7fe048549640 1 --2- 172.21.15.5:0/2273044553 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe02c067660 0x7fe02c069b20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe0300ae000 tx=0x7fe0300a7070 comp rx=0 tx=0).stop 2024-03-18T15:33:53.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.821+0000 7fe048549640 1 -- 172.21.15.5:0/2273044553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0400ae9c0 msgr2=0x7fe0401e3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:53.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.821+0000 7fe048549640 1 --2- 172.21.15.5:0/2273044553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0400ae9c0 0x7fe0401e3a80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe03800b410 tx=0x7fe03800b8d0 comp rx=0 tx=0).stop 2024-03-18T15:33:53.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.821+0000 7fe048549640 1 -- 172.21.15.5:0/2273044553 shutdown_connections 2024-03-18T15:33:53.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.821+0000 7fe048549640 1 --2- 172.21.15.5:0/2273044553 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe02c067660 0x7fe02c069b20 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:53.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.821+0000 7fe048549640 1 --2- 172.21.15.5:0/2273044553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0400ae9c0 0x7fe0401e3a80 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:53.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.821+0000 7fe048549640 1 --2- 172.21.15.5:0/2273044553 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0400ae0f0 0x7fe040149530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:53.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.821+0000 7fe048549640 1 --2- 172.21.15.5:0/2273044553 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe04006d2d0 0x7fe04013d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:53.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.821+0000 7fe048549640 1 -- 172.21.15.5:0/2273044553 >> 172.21.15.5:0/2273044553 conn(0x7fe0400b5670 msgr2=0x7fe04006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:53.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.821+0000 7fe048549640 1 -- 172.21.15.5:0/2273044553 shutdown_connections 2024-03-18T15:33:53.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.821+0000 7fe048549640 1 -- 172.21.15.5:0/2273044553 wait complete. 2024-03-18T15:33:53.839 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:33:53.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.957+0000 7f024ab72640 1 Processor -- start 2024-03-18T15:33:53.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.957+0000 7f024ab72640 1 -- start start 2024-03-18T15:33:53.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.957+0000 7f024ab72640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f02440aec00 0x7f02440aefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:53.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.957+0000 7f024ab72640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0244143bd0 0x7f0244143ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:53.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.957+0000 7f024ab72640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02440add60 con 0x7f0244147060 2024-03-18T15:33:53.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.957+0000 7f024ab72640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f02440ae090 con 0x7f02440aec00 2024-03-18T15:33:53.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.957+0000 7f024ab72640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f02440b0350 con 0x7f0244143bd0 2024-03-18T15:33:53.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.957+0000 7f024890e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f02440aec00 0x7f02440aefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:53.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.961+0000 7f024890e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f02440aec00 0x7f02440aefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43018/0 (socket says 172.21.15.5:43018) 2024-03-18T15:33:53.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.961+0000 7f024890e640 1 -- 172.21.15.5:0/3749536918 learned_addr learned my addr 172.21.15.5:0/3749536918 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:53.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.961+0000 7f024910f640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f0244147060 0x7f0244147460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46920/0 (socket says 172.21.15.5:46920) 2024-03-18T15:33:53.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.961+0000 7f023bfff640 1 --2- 172.21.15.5:0/3749536918 >> v2:172.21.15.80:3300/0 conn(0x7f0244143bd0 0x7f0244143ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:53.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.961+0000 7f024890e640 1 -- 172.21.15.5:0/3749536918 >> v2:172.21.15.80:3300/0 conn(0x7f0244143bd0 msgr2=0x7f0244143ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:53.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.961+0000 7f024890e640 1 --2- 172.21.15.5:0/3749536918 >> v2:172.21.15.80:3300/0 conn(0x7f0244143bd0 0x7f0244143ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:53.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.961+0000 7f024890e640 1 -- 172.21.15.5:0/3749536918 >> v1:172.21.15.5:6789/0 conn(0x7f0244147060 legacy=0x7f0244147460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:53.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.961+0000 7f024890e640 1 -- 172.21.15.5:0/3749536918 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02440b0630 con 0x7f02440aec00 2024-03-18T15:33:53.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.961+0000 7f024890e640 1 --2- 172.21.15.5:0/3749536918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02440aec00 0x7f02440aefe0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0244072b40 tx=0x7f024001c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a34323b58cb0e79f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:53.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.961+0000 7f023b7fe640 1 -- 172.21.15.5:0/3749536918 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02400b0070 con 0x7f02440aec00 2024-03-18T15:33:53.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.961+0000 7f023b7fe640 1 -- 172.21.15.5:0/3749536918 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0240004230 con 0x7f02440aec00 2024-03-18T15:33:53.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.961+0000 7f023b7fe640 1 -- 172.21.15.5:0/3749536918 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0240003620 con 0x7f02440aec00 2024-03-18T15:33:53.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.961+0000 7f024ab72640 1 -- 172.21.15.5:0/3749536918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02440aec00 msgr2=0x7f02440aefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:53.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.961+0000 7f024ab72640 1 --2- 172.21.15.5:0/3749536918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02440aec00 0x7f02440aefe0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0244072b40 tx=0x7f024001c3b0 comp rx=0 tx=0).stop 2024-03-18T15:33:53.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.961+0000 7f024ab72640 1 -- 172.21.15.5:0/3749536918 shutdown_connections 2024-03-18T15:33:53.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.961+0000 7f024ab72640 1 --2- 172.21.15.5:0/3749536918 >> v2:172.21.15.80:3300/0 conn(0x7f0244143bd0 0x7f0244143ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:53.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.961+0000 7f024ab72640 1 --2- 172.21.15.5:0/3749536918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02440aec00 0x7f02440aefe0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:53.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.961+0000 7f024ab72640 1 -- 172.21.15.5:0/3749536918 >> 172.21.15.5:0/3749536918 conn(0x7f02440b5670 msgr2=0x7f02440b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:53.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.961+0000 7f024ab72640 1 -- 172.21.15.5:0/3749536918 shutdown_connections 2024-03-18T15:33:53.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.961+0000 7f024ab72640 1 -- 172.21.15.5:0/3749536918 wait complete. 2024-03-18T15:33:53.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.965+0000 7f024ab72640 1 Processor -- start 2024-03-18T15:33:53.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.965+0000 7f024ab72640 1 -- start start 2024-03-18T15:33:53.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.965+0000 7f024ab72640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02440aec00 0x7f0244146530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:53.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.965+0000 7f024ab72640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0244143bd0 0x7f024406fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:53.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.965+0000 7f024ab72640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0244147060 0x7f02441dbc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:53.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.965+0000 7f024ab72640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f02440ae4f0 con 0x7f02440aec00 2024-03-18T15:33:53.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.965+0000 7f024ab72640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f02440b51c0 con 0x7f0244143bd0 2024-03-18T15:33:53.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.965+0000 7f024ab72640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f0244147b70 con 0x7f0244147060 2024-03-18T15:33:53.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.965+0000 7f024890e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02440aec00 0x7f0244146530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:53.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.965+0000 7f024890e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02440aec00 0x7f0244146530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43030/0 (socket says 172.21.15.5:43030) 2024-03-18T15:33:53.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.965+0000 7f024890e640 1 -- 172.21.15.5:0/4214907625 learned_addr learned my addr 172.21.15.5:0/4214907625 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:53.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.965+0000 7f023bfff640 1 --2- 172.21.15.5:0/4214907625 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0244143bd0 0x7f024406fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:53.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.965+0000 7f024910f640 1 --2- 172.21.15.5:0/4214907625 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0244147060 0x7f02441dbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:53.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.965+0000 7f024890e640 1 -- 172.21.15.5:0/4214907625 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0244147060 msgr2=0x7f02441dbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:53.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.965+0000 7f024890e640 1 --2- 172.21.15.5:0/4214907625 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0244147060 0x7f02441dbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:53.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.965+0000 7f024890e640 1 -- 172.21.15.5:0/4214907625 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0244143bd0 msgr2=0x7f024406fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:53.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.965+0000 7f024890e640 1 --2- 172.21.15.5:0/4214907625 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0244143bd0 0x7f024406fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:53.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.965+0000 7f024890e640 1 -- 172.21.15.5:0/4214907625 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02440b26d0 con 0x7f02440aec00 2024-03-18T15:33:53.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.965+0000 7f024910f640 1 --2- 172.21.15.5:0/4214907625 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0244147060 0x7f02441dbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:33:53.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.965+0000 7f024890e640 1 --2- 172.21.15.5:0/4214907625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02440aec00 0x7f0244146530 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0240019fe0 tx=0x7f0240019150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:53.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.965+0000 7f02397fa640 1 -- 172.21.15.5:0/4214907625 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02400b0040 con 0x7f02440aec00 2024-03-18T15:33:53.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.965+0000 7f02397fa640 1 -- 172.21.15.5:0/4214907625 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02400ab440 con 0x7f02440aec00 2024-03-18T15:33:53.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.965+0000 7f024ab72640 1 -- 172.21.15.5:0/4214907625 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0244071510 con 0x7f02440aec00 2024-03-18T15:33:53.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.965+0000 7f02397fa640 1 -- 172.21.15.5:0/4214907625 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02400ab830 con 0x7f02440aec00 2024-03-18T15:33:53.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.965+0000 7f024ab72640 1 -- 172.21.15.5:0/4214907625 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02440b2bd0 con 0x7f02440aec00 2024-03-18T15:33:53.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.965+0000 7f02397fa640 1 -- 172.21.15.5:0/4214907625 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f02400bd020 con 0x7f02440aec00 2024-03-18T15:33:53.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.965+0000 7f024ab72640 1 -- 172.21.15.5:0/4214907625 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0244072230 con 0x7f02440aec00 2024-03-18T15:33:53.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.969+0000 7f02397fa640 1 --2- 172.21.15.5:0/4214907625 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0230067690 0x7f0230069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:53.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.969+0000 7f023bfff640 1 --2- 172.21.15.5:0/4214907625 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0230067690 0x7f0230069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:53.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.969+0000 7f02397fa640 1 -- 172.21.15.5:0/4214907625 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f024011f7f0 con 0x7f02440aec00 2024-03-18T15:33:53.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.969+0000 7f023bfff640 1 --2- 172.21.15.5:0/4214907625 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0230067690 0x7f0230069b50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f024409e3e0 tx=0x7f0234008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:53.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:53.973+0000 7f02397fa640 1 -- 172.21.15.5:0/4214907625 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f02400c4020 con 0x7f02440aec00 2024-03-18T15:33:54.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.169+0000 7f024ab72640 1 -- 172.21.15.5:0/4214907625 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f02440b78d0 con 0x7f02440aec00 2024-03-18T15:33:54.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.173+0000 7f02397fa640 1 -- 172.21.15.5:0/4214907625 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+3521 (secure 0 0 0) 0x7f0240016030 con 0x7f02440aec00 2024-03-18T15:33:54.174 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:33:54.174 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/629589896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":629589896},{"type":"v1","addr":"172.21.15.5:6835","nonce":629589896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-18T15:33:42.338926+0000","modified":"2024-03-18T15:33:45.376646+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6833/1236603350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":1236603350},{"type":"v1","addr":"172.21.15.80:6833","nonce":1236603350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4272,"qdb_cluster":[4272]},"id":1}]} 2024-03-18T15:33:54.175 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 8 2024-03-18T15:33:54.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.173+0000 7f024ab72640 1 -- 172.21.15.5:0/4214907625 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0230067690 msgr2=0x7f0230069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:54.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.173+0000 7f024ab72640 1 --2- 172.21.15.5:0/4214907625 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0230067690 0x7f0230069b50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f024409e3e0 tx=0x7f0234008040 comp rx=0 tx=0).stop 2024-03-18T15:33:54.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.173+0000 7f024ab72640 1 -- 172.21.15.5:0/4214907625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02440aec00 msgr2=0x7f0244146530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:54.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.173+0000 7f024ab72640 1 --2- 172.21.15.5:0/4214907625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02440aec00 0x7f0244146530 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0240019fe0 tx=0x7f0240019150 comp rx=0 tx=0).stop 2024-03-18T15:33:54.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.173+0000 7f024ab72640 1 -- 172.21.15.5:0/4214907625 shutdown_connections 2024-03-18T15:33:54.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.173+0000 7f024ab72640 1 --2- 172.21.15.5:0/4214907625 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0230067690 0x7f0230069b50 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:54.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.173+0000 7f024ab72640 1 --2- 172.21.15.5:0/4214907625 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0244147060 0x7f02441dbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:54.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.173+0000 7f024ab72640 1 --2- 172.21.15.5:0/4214907625 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0244143bd0 0x7f024406fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:54.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.173+0000 7f024ab72640 1 --2- 172.21.15.5:0/4214907625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02440aec00 0x7f0244146530 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:54.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.173+0000 7f024ab72640 1 -- 172.21.15.5:0/4214907625 >> 172.21.15.5:0/4214907625 conn(0x7f02440b5670 msgr2=0x7f0244141a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:54.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.173+0000 7f024ab72640 1 -- 172.21.15.5:0/4214907625 shutdown_connections 2024-03-18T15:33:54.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.173+0000 7f024ab72640 1 -- 172.21.15.5:0/4214907625 wait complete. 2024-03-18T15:33:54.395 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-18T15:33:42.338926+0000', 'modified': '2024-03-18T15:33:45.376646+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4272}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4272': {'gid': 4272, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.80:6833/1236603350', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.80:6832', 'nonce': 1236603350}, {'type': 'v1', 'addr': '172.21.15.80:6833', 'nonce': 1236603350}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4272, 'qdb_cluster': [4272]} 2024-03-18T15:33:54.396 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-18T15:33:54.396 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-18T15:33:54.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.525+0000 7f5c3b6ee640 1 Processor -- start 2024-03-18T15:33:54.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.525+0000 7f5c3b6ee640 1 -- start start 2024-03-18T15:33:54.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.525+0000 7f5c3b6ee640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5c3406d2d0 0x7f5c340b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:54.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.525+0000 7f5c3b6ee640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c3413fb10 con 0x7f5c3413f260 2024-03-18T15:33:54.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.525+0000 7f5c3b6ee640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c340b0ea0 con 0x7f5c3413f6a0 2024-03-18T15:33:54.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.525+0000 7f5c3b6ee640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f5c340728c0 con 0x7f5c3406d2d0 2024-03-18T15:33:54.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.525+0000 7f5c38c89640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f5c3413f260 0x7f5c341453a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46928/0 (socket says 172.21.15.5:46928) 2024-03-18T15:33:54.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.525+0000 7f5c38c89640 1 -- 172.21.15.5:0/2001470110 learned_addr learned my addr 172.21.15.5:0/2001470110 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:33:54.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.525+0000 7f5c3948a640 1 --2- 172.21.15.5:0/2001470110 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5c3406d2d0 0x7f5c340b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:54.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.525+0000 7f5c2bfff640 1 -- 172.21.15.5:0/2001470110 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3738614922 0 0) 0x7f5c3413fb10 con 0x7f5c3413f260 2024-03-18T15:33:54.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.525+0000 7f5c2bfff640 1 -- 172.21.15.5:0/2001470110 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c20003530 con 0x7f5c3413f260 2024-03-18T15:33:54.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.525+0000 7f5c2bfff640 1 -- 172.21.15.5:0/2001470110 <== mon.1 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 256328322 0 0) 0x7f5c340b0ea0 con 0x7f5c3413f6a0 2024-03-18T15:33:54.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.525+0000 7f5c2bfff640 1 -- 172.21.15.5:0/2001470110 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c3413fb10 con 0x7f5c3413f6a0 2024-03-18T15:33:54.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.525+0000 7f5c2bfff640 1 -- 172.21.15.5:0/2001470110 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 431677482 0 0) 0x7f5c20003530 con 0x7f5c3413f260 2024-03-18T15:33:54.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.525+0000 7f5c3948a640 1 -- 172.21.15.5:0/2001470110 >> v1:172.21.15.80:6789/0 conn(0x7f5c3413f6a0 legacy=0x7f5c34145ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:54.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.525+0000 7f5c3948a640 1 -- 172.21.15.5:0/2001470110 >> v1:172.21.15.5:6789/0 conn(0x7f5c3413f260 legacy=0x7f5c341453a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:54.530 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.525+0000 7f5c3948a640 1 -- 172.21.15.5:0/2001470110 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c3406d6b0 con 0x7f5c3406d2d0 2024-03-18T15:33:54.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.529+0000 7f5c3948a640 1 --2- 172.21.15.5:0/2001470110 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5c3406d2d0 0x7f5c340b23a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5c300097f0 tx=0x7f5c3001c340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=580045c813cae4ad server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:54.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.529+0000 7f5c2bfff640 1 -- 172.21.15.5:0/2001470110 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c300b0070 con 0x7f5c3406d2d0 2024-03-18T15:33:54.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.529+0000 7f5c2bfff640 1 -- 172.21.15.5:0/2001470110 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c300ab400 con 0x7f5c3406d2d0 2024-03-18T15:33:54.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.529+0000 7f5c2bfff640 1 -- 172.21.15.5:0/2001470110 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c24001680 con 0x7f5c3406d2d0 2024-03-18T15:33:54.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.529+0000 7f5c3b6ee640 1 -- 172.21.15.5:0/2001470110 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5c3406d2d0 msgr2=0x7f5c340b23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:54.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.529+0000 7f5c3b6ee640 1 --2- 172.21.15.5:0/2001470110 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5c3406d2d0 0x7f5c340b23a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5c300097f0 tx=0x7f5c3001c340 comp rx=0 tx=0).stop 2024-03-18T15:33:54.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.529+0000 7f5c3b6ee640 1 -- 172.21.15.5:0/2001470110 shutdown_connections 2024-03-18T15:33:54.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.529+0000 7f5c3b6ee640 1 --2- 172.21.15.5:0/2001470110 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5c3406d2d0 0x7f5c340b23a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:54.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.529+0000 7f5c3b6ee640 1 -- 172.21.15.5:0/2001470110 >> 172.21.15.5:0/2001470110 conn(0x7f5c340b5670 msgr2=0x7f5c340b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:54.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.529+0000 7f5c3b6ee640 1 -- 172.21.15.5:0/2001470110 shutdown_connections 2024-03-18T15:33:54.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.529+0000 7f5c3b6ee640 1 -- 172.21.15.5:0/2001470110 wait complete. 2024-03-18T15:33:54.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.533+0000 7f5c3b6ee640 1 Processor -- start 2024-03-18T15:33:54.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.533+0000 7f5c3b6ee640 1 -- start start 2024-03-18T15:33:54.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.533+0000 7f5c3b6ee640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5c3406d2d0 0x7f5c34071aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:54.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.533+0000 7f5c3b6ee640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5c3413f260 0x7f5c341e1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:54.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.533+0000 7f5c3b6ee640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c3413f6a0 0x7f5c341e1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:54.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.533+0000 7f5c3b6ee640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5c340b01f0 con 0x7f5c3413f6a0 2024-03-18T15:33:54.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.533+0000 7f5c3b6ee640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f5c340b0070 con 0x7f5c3406d2d0 2024-03-18T15:33:54.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.533+0000 7f5c3b6ee640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f5c340b07e0 con 0x7f5c3413f260 2024-03-18T15:33:54.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.533+0000 7f5c39c8b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c3413f6a0 0x7f5c341e1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:54.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.533+0000 7f5c39c8b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c3413f6a0 0x7f5c341e1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43042/0 (socket says 172.21.15.5:43042) 2024-03-18T15:33:54.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.533+0000 7f5c39c8b640 1 -- 172.21.15.5:0/2139320552 learned_addr learned my addr 172.21.15.5:0/2139320552 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:54.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.533+0000 7f5c38c89640 1 --2- 172.21.15.5:0/2139320552 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5c3413f260 0x7f5c341e1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:54.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.533+0000 7f5c3948a640 1 --2- 172.21.15.5:0/2139320552 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5c3406d2d0 0x7f5c34071aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:54.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.533+0000 7f5c39c8b640 1 -- 172.21.15.5:0/2139320552 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5c3413f260 msgr2=0x7f5c341e1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:54.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.533+0000 7f5c39c8b640 1 --2- 172.21.15.5:0/2139320552 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5c3413f260 0x7f5c341e1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:54.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.533+0000 7f5c39c8b640 1 -- 172.21.15.5:0/2139320552 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5c3406d2d0 msgr2=0x7f5c34071aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:54.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.533+0000 7f5c39c8b640 1 --2- 172.21.15.5:0/2139320552 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5c3406d2d0 0x7f5c34071aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:54.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.533+0000 7f5c39c8b640 1 -- 172.21.15.5:0/2139320552 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c34148a70 con 0x7f5c3413f6a0 2024-03-18T15:33:54.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.533+0000 7f5c3948a640 1 --2- 172.21.15.5:0/2139320552 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5c3406d2d0 0x7f5c34071aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:33:54.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.533+0000 7f5c39c8b640 1 --2- 172.21.15.5:0/2139320552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c3413f6a0 0x7f5c341e1790 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c00c730 tx=0x7f5c2c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:54.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.533+0000 7f5c29ffb640 1 -- 172.21.15.5:0/2139320552 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c003b80 con 0x7f5c3413f6a0 2024-03-18T15:33:54.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.533+0000 7f5c29ffb640 1 -- 172.21.15.5:0/2139320552 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c2c003d20 con 0x7f5c3413f6a0 2024-03-18T15:33:54.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.533+0000 7f5c3b6ee640 1 -- 172.21.15.5:0/2139320552 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c34148d50 con 0x7f5c3413f6a0 2024-03-18T15:33:54.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.533+0000 7f5c29ffb640 1 -- 172.21.15.5:0/2139320552 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c00aa90 con 0x7f5c3413f6a0 2024-03-18T15:33:54.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.533+0000 7f5c3b6ee640 1 -- 172.21.15.5:0/2139320552 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c34148790 con 0x7f5c3413f6a0 2024-03-18T15:33:54.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.533+0000 7f5c29ffb640 1 -- 172.21.15.5:0/2139320552 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f5c2c00ac30 con 0x7f5c3413f6a0 2024-03-18T15:33:54.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.533+0000 7f5c29ffb640 1 --2- 172.21.15.5:0/2139320552 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5c18067660 0x7f5c18069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:54.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.533+0000 7f5c29ffb640 1 -- 172.21.15.5:0/2139320552 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f5c2c085740 con 0x7f5c3413f6a0 2024-03-18T15:33:54.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.533+0000 7f5c3948a640 1 --2- 172.21.15.5:0/2139320552 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5c18067660 0x7f5c18069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:54.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.537+0000 7f5c3948a640 1 --2- 172.21.15.5:0/2139320552 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5c18067660 0x7f5c18069b20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5c300ad000 tx=0x7f5c300a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:54.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.537+0000 7f5c3e870000 1 Processor -- start 2024-03-18T15:33:54.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.537+0000 7f5c3e870000 1 -- start start 2024-03-18T15:33:54.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.537+0000 7f5c3e870000 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55da165724a0 0x55da16572880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:54.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.537+0000 7f5c3e870000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55da16572dc0 0x55da1659d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:54.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.537+0000 7f5c3e870000 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da1659d6c0 0x55da1659daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:54.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.537+0000 7f5c3e870000 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55da163bb540 con 0x55da1659d6c0 2024-03-18T15:33:54.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.537+0000 7f5c3e870000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55da16342b00 con 0x55da16572dc0 2024-03-18T15:33:54.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.537+0000 7f5c3e870000 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x55da163b75e0 con 0x55da165724a0 2024-03-18T15:33:54.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.537+0000 7f5c39c8b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da1659d6c0 0x55da1659daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:54.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.537+0000 7f5c39c8b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da1659d6c0 0x55da1659daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43058/0 (socket says 172.21.15.5:43058) 2024-03-18T15:33:54.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.537+0000 7f5c39c8b640 1 -- 172.21.15.5:0/2108376752 learned_addr learned my addr 172.21.15.5:0/2108376752 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:54.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.537+0000 7f5c3948a640 1 --2- 172.21.15.5:0/2108376752 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55da165724a0 0x55da16572880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:54.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.537+0000 7f5c38c89640 1 --2- 172.21.15.5:0/2108376752 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55da16572dc0 0x55da1659d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:54.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.537+0000 7f5c39c8b640 1 -- 172.21.15.5:0/2108376752 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55da165724a0 msgr2=0x55da16572880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:54.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.537+0000 7f5c39c8b640 1 --2- 172.21.15.5:0/2108376752 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55da165724a0 0x55da16572880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:54.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.537+0000 7f5c39c8b640 1 -- 172.21.15.5:0/2108376752 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55da16572dc0 msgr2=0x55da1659d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:54.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.537+0000 7f5c39c8b640 1 --2- 172.21.15.5:0/2108376752 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55da16572dc0 0x55da1659d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:54.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.537+0000 7f5c39c8b640 1 -- 172.21.15.5:0/2108376752 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da163786a0 con 0x55da1659d6c0 2024-03-18T15:33:54.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.537+0000 7f5c39c8b640 1 --2- 172.21.15.5:0/2108376752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da1659d6c0 0x55da1659daa0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c05df30 tx=0x7f5c2c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=513c0367b1184c57 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:54.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.537+0000 7f5c1effd640 1 -- 172.21.15.5:0/2108376752 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c082070 con 0x55da1659d6c0 2024-03-18T15:33:54.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.537+0000 7f5c1effd640 1 -- 172.21.15.5:0/2108376752 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c2c055d50 con 0x55da1659d6c0 2024-03-18T15:33:54.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.537+0000 7f5c1effd640 1 -- 172.21.15.5:0/2108376752 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c059ea0 con 0x55da1659d6c0 2024-03-18T15:33:54.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.537+0000 7f5c3e870000 1 -- 172.21.15.5:0/2108376752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da1659d6c0 msgr2=0x55da1659daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:54.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.537+0000 7f5c3e870000 1 --2- 172.21.15.5:0/2108376752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da1659d6c0 0x55da1659daa0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c05df30 tx=0x7f5c2c011000 comp rx=0 tx=0).stop 2024-03-18T15:33:54.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.537+0000 7f5c3e870000 1 -- 172.21.15.5:0/2108376752 shutdown_connections 2024-03-18T15:33:54.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.537+0000 7f5c3e870000 1 --2- 172.21.15.5:0/2108376752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da1659d6c0 0x55da1659daa0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:54.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.537+0000 7f5c3e870000 1 --2- 172.21.15.5:0/2108376752 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55da16572dc0 0x55da1659d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:54.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.537+0000 7f5c3e870000 1 --2- 172.21.15.5:0/2108376752 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55da165724a0 0x55da16572880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:54.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.537+0000 7f5c3e870000 1 -- 172.21.15.5:0/2108376752 >> 172.21.15.5:0/2108376752 conn(0x55da16598f30 msgr2=0x55da1659b320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:54.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.537+0000 7f5c3e870000 1 -- 172.21.15.5:0/2108376752 shutdown_connections 2024-03-18T15:33:54.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.537+0000 7f5c3e870000 1 -- 172.21.15.5:0/2108376752 wait complete. 2024-03-18T15:33:54.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.537+0000 7f5c3e870000 1 -- start start 2024-03-18T15:33:54.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.537+0000 7f5c3e870000 1 Processor -- start 2024-03-18T15:33:54.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.537+0000 7f5c3e870000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55da165724a0 0x55da1656be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:54.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.537+0000 7f5c3e870000 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55da16572dc0 0x55da1656c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:54.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.541+0000 7f5c3e870000 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da1659d6c0 0x55da1656c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:54.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.541+0000 7f5c3e870000 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55da163bfb70 con 0x55da1659d6c0 2024-03-18T15:33:54.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.541+0000 7f5c3e870000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55da16379850 con 0x55da165724a0 2024-03-18T15:33:54.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.541+0000 7f5c3e870000 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x55da1659f020 con 0x55da16572dc0 2024-03-18T15:33:54.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.541+0000 7f5c3e870000 10 client.0 _ll_register_callbacks cb 0x55da16558220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:33:54.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.541+0000 7f5c39c8b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da1659d6c0 0x55da1656c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:54.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.541+0000 7f5c39c8b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da1659d6c0 0x55da1656c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43066/0 (socket says 172.21.15.5:43066) 2024-03-18T15:33:54.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.541+0000 7f5c39c8b640 1 -- 172.21.15.5:0/960836691 learned_addr learned my addr 172.21.15.5:0/960836691 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:54.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.541+0000 7f5c3948a640 1 --2- 172.21.15.5:0/960836691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55da165724a0 0x55da1656be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:54.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.541+0000 7f5c38c89640 1 --2- 172.21.15.5:0/960836691 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55da16572dc0 0x55da1656c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:54.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.541+0000 7f5c39c8b640 1 -- 172.21.15.5:0/960836691 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55da16572dc0 msgr2=0x55da1656c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:54.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.541+0000 7f5c39c8b640 1 --2- 172.21.15.5:0/960836691 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55da16572dc0 0x55da1656c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:54.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.541+0000 7f5c39c8b640 1 -- 172.21.15.5:0/960836691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55da165724a0 msgr2=0x55da1656be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:54.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.541+0000 7f5c39c8b640 1 --2- 172.21.15.5:0/960836691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55da165724a0 0x55da1656be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:54.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.541+0000 7f5c39c8b640 1 -- 172.21.15.5:0/960836691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da16376900 con 0x55da1659d6c0 2024-03-18T15:33:54.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.541+0000 7f5c39c8b640 1 --2- 172.21.15.5:0/960836691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da1659d6c0 0x55da1656c8d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c05df30 tx=0x7f5c2c001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab2f4628ac93bdfd server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:54.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.541+0000 7f5c13fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-03-18T15:33:54.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.541+0000 7f5c13fff640 1 -- 172.21.15.5:0/960836691 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c082040 con 0x55da1659d6c0 2024-03-18T15:33:54.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.541+0000 7f5c13fff640 1 -- 172.21.15.5:0/960836691 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f5c2c054050 con 0x55da1659d6c0 2024-03-18T15:33:54.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.541+0000 7f5c3e870000 1 -- 172.21.15.5:0/960836691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c2c055d50 con 0x55da1659d6c0 2024-03-18T15:33:54.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.541+0000 7f5c13fff640 1 -- 172.21.15.5:0/960836691 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f5c2c07c050 con 0x55da1659d6c0 2024-03-18T15:33:54.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.541+0000 7f5c3e870000 10 client.4629 fetch_fsmap learned FSMap version 8 2024-03-18T15:33:54.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.541+0000 7f5c3e870000 1 -- 172.21.15.5:0/960836691 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55da1643c940 con 0x55da1659d6c0 2024-03-18T15:33:54.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.541+0000 7f5c13fff640 1 -- 172.21.15.5:0/960836691 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c2c055d50 con 0x55da1659d6c0 2024-03-18T15:33:54.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.541+0000 7f5c13fff640 1 -- 172.21.15.5:0/960836691 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 8) ==== 2284+0+0 (secure 0 0 0) 0x7f5c2c0523c0 con 0x55da1659d6c0 2024-03-18T15:33:54.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.541+0000 7f5c3e870000 10 client.4629 fetch_fsmap finished waiting for FSMap version 8 2024-03-18T15:33:54.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.541+0000 7f5c3e870000 10 client.4629 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-18T15:33:54.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.541+0000 7f5c3e870000 20 client.4629 populate_metadata read hostname 'smithi005' 2024-03-18T15:33:54.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.541+0000 7f5c3e870000 1 --2- 172.21.15.5:0/960836691 >> [v2:172.21.15.80:6832/1236603350,v1:172.21.15.80:6833/1236603350] conn(0x55da165b0560 0x55da165b2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:54.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.541+0000 7f5c3e870000 4 client.4629 mds_command: new command op to 4272 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:33:54.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.541+0000 7f5c3e870000 1 -- 172.21.15.5:0/960836691 --> [v2:172.21.15.80:6832/1236603350,v1:172.21.15.80:6833/1236603350] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da16356bc0 con 0x55da165b0560 2024-03-18T15:33:54.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.541+0000 7f5c3948a640 1 --2- 172.21.15.5:0/960836691 >> [v2:172.21.15.80:6832/1236603350,v1:172.21.15.80:6833/1236603350] conn(0x55da165b0560 0x55da165b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:54.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.541+0000 7f5c3948a640 1 --2- 172.21.15.5:0/960836691 >> [v2:172.21.15.80:6832/1236603350,v1:172.21.15.80:6833/1236603350] conn(0x55da165b0560 0x55da165b2950 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed035bc43af197e8 server_cookie=ae8d40d93d0689f6 in_seq=0 out_seq=0 2024-03-18T15:33:54.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.541+0000 7f5c13fff640 10 client.4629 ms_handle_connect on v2:172.21.15.80:6832/1236603350 2024-03-18T15:33:54.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.545+0000 7f5c13fff640 1 -- 172.21.15.5:0/960836691 <== mds.0 v2:172.21.15.80:6832/1236603350 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x55da16356bc0 con 0x55da165b0560 2024-03-18T15:33:54.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.545+0000 7f5c13fff640 10 client.4629 handle_command_reply: tid=0 2024-03-18T15:33:54.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.545+0000 7f5c3e870000 1 client.4629 shutdown 2024-03-18T15:33:54.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.545+0000 7f5c3e870000 1 -- 172.21.15.5:0/960836691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da1659d6c0 msgr2=0x55da1656c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:54.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.545+0000 7f5c3e870000 1 --2- 172.21.15.5:0/960836691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da1659d6c0 0x55da1656c8d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c05df30 tx=0x7f5c2c001260 comp rx=0 tx=0).stop 2024-03-18T15:33:54.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.545+0000 7f5c3e870000 1 -- 172.21.15.5:0/960836691 shutdown_connections 2024-03-18T15:33:54.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.545+0000 7f5c3e870000 1 --2- 172.21.15.5:0/960836691 >> [v2:172.21.15.80:6832/1236603350,v1:172.21.15.80:6833/1236603350] conn(0x55da165b0560 0x55da165b2950 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:54.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.545+0000 7f5c3e870000 1 --2- 172.21.15.5:0/960836691 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da1659d6c0 0x55da1656c8d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:54.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.545+0000 7f5c3e870000 1 --2- 172.21.15.5:0/960836691 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55da16572dc0 0x55da1656c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:54.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.545+0000 7f5c3e870000 1 --2- 172.21.15.5:0/960836691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55da165724a0 0x55da1656be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:54.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.545+0000 7f5c13fff640 0 client.4629 ms_handle_reset on v2:172.21.15.80:6832/1236603350 2024-03-18T15:33:54.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.545+0000 7f5c3e870000 1 -- 172.21.15.5:0/960836691 >> 172.21.15.5:0/960836691 conn(0x55da16598f30 msgr2=0x55da1659b320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:54.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.545+0000 7f5c3e870000 1 -- 172.21.15.5:0/960836691 shutdown_connections 2024-03-18T15:33:54.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.545+0000 7f5c3e870000 1 -- 172.21.15.5:0/960836691 wait complete. 2024-03-18T15:33:54.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.545+0000 7f5c3e870000 20 client.4629 trim_cache size 0 max 16384 2024-03-18T15:33:54.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.561+0000 7f5c3e870000 1 Processor -- start 2024-03-18T15:33:54.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.561+0000 7f5c3e870000 1 -- start start 2024-03-18T15:33:54.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.561+0000 7f5c3e870000 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55da165724a0 0x55da1659c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:54.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.561+0000 7f5c3e870000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55da16572dc0 0x55da1660a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:54.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.561+0000 7f5c3e870000 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da1659d6c0 0x55da1660ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:54.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.561+0000 7f5c3e870000 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5c341ea190 con 0x55da1659d6c0 2024-03-18T15:33:54.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.561+0000 7f5c3e870000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f5c34070770 con 0x55da16572dc0 2024-03-18T15:33:54.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.561+0000 7f5c3e870000 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x55da1643c4b0 con 0x55da165724a0 2024-03-18T15:33:54.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.561+0000 7f5c39c8b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da1659d6c0 0x55da1660ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:54.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.561+0000 7f5c39c8b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da1659d6c0 0x55da1660ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43078/0 (socket says 172.21.15.5:43078) 2024-03-18T15:33:54.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.561+0000 7f5c39c8b640 1 -- 172.21.15.5:0/3681360718 learned_addr learned my addr 172.21.15.5:0/3681360718 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:54.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.561+0000 7f5c3948a640 1 --2- 172.21.15.5:0/3681360718 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55da165724a0 0x55da1659c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:54.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.561+0000 7f5c38c89640 1 --2- 172.21.15.5:0/3681360718 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55da16572dc0 0x55da1660a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:54.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.561+0000 7f5c39c8b640 1 -- 172.21.15.5:0/3681360718 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55da165724a0 msgr2=0x55da1659c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:54.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.561+0000 7f5c39c8b640 1 --2- 172.21.15.5:0/3681360718 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55da165724a0 0x55da1659c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:54.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.561+0000 7f5c39c8b640 1 -- 172.21.15.5:0/3681360718 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55da16572dc0 msgr2=0x55da1660a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:54.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.561+0000 7f5c39c8b640 1 --2- 172.21.15.5:0/3681360718 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55da16572dc0 0x55da1660a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:54.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.561+0000 7f5c39c8b640 1 -- 172.21.15.5:0/3681360718 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da16413870 con 0x55da1659d6c0 2024-03-18T15:33:54.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.561+0000 7f5c38c89640 1 --2- 172.21.15.5:0/3681360718 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55da16572dc0 0x55da1660a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:33:54.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.561+0000 7f5c39c8b640 1 --2- 172.21.15.5:0/3681360718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da1659d6c0 0x55da1660ad20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c07d260 tx=0x7f5c2c06e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=673739ea189b2dc2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:54.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.561+0000 7f5c1e7fc640 1 -- 172.21.15.5:0/3681360718 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c082040 con 0x55da1659d6c0 2024-03-18T15:33:54.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.561+0000 7f5c1e7fc640 1 -- 172.21.15.5:0/3681360718 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c2c055d50 con 0x55da1659d6c0 2024-03-18T15:33:54.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.561+0000 7f5c1e7fc640 1 -- 172.21.15.5:0/3681360718 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2c059ea0 con 0x55da1659d6c0 2024-03-18T15:33:54.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.565+0000 7f5c3e870000 1 -- 172.21.15.5:0/3681360718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da1659d6c0 msgr2=0x55da1660ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:54.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.565+0000 7f5c3e870000 1 --2- 172.21.15.5:0/3681360718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da1659d6c0 0x55da1660ad20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c07d260 tx=0x7f5c2c06e000 comp rx=0 tx=0).stop 2024-03-18T15:33:54.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.565+0000 7f5c3e870000 1 -- 172.21.15.5:0/3681360718 shutdown_connections 2024-03-18T15:33:54.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.565+0000 7f5c3e870000 1 --2- 172.21.15.5:0/3681360718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da1659d6c0 0x55da1660ad20 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:54.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.565+0000 7f5c3e870000 1 --2- 172.21.15.5:0/3681360718 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55da16572dc0 0x55da1660a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:54.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.565+0000 7f5c3e870000 1 --2- 172.21.15.5:0/3681360718 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55da165724a0 0x55da1659c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:54.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.565+0000 7f5c3e870000 1 -- 172.21.15.5:0/3681360718 >> 172.21.15.5:0/3681360718 conn(0x55da16598f30 msgr2=0x55da165b7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:54.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.565+0000 7f5c3e870000 1 -- 172.21.15.5:0/3681360718 shutdown_connections 2024-03-18T15:33:54.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.565+0000 7f5c3e870000 1 -- 172.21.15.5:0/3681360718 wait complete. 2024-03-18T15:33:54.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.565+0000 7f5c3e870000 1 -- start start 2024-03-18T15:33:54.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.565+0000 7f5c3e870000 1 Processor -- start 2024-03-18T15:33:54.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.565+0000 7f5c3e870000 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da165724a0 0x55da165e2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:54.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.565+0000 7f5c3e870000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55da16572dc0 0x55da165e2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:54.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.565+0000 7f5c3e870000 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55da1659d6c0 0x55da165e2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:54.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.565+0000 7f5c3e870000 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55da163537b0 con 0x55da165724a0 2024-03-18T15:33:54.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.565+0000 7f5c3e870000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55da16375f50 con 0x55da16572dc0 2024-03-18T15:33:54.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.565+0000 7f5c3e870000 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x55da163ba850 con 0x55da1659d6c0 2024-03-18T15:33:54.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.565+0000 7f5c3e870000 10 client.0 _ll_register_callbacks cb 0x7f5c2c080070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:33:54.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.565+0000 7f5c3948a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da165724a0 0x55da165e2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:54.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.565+0000 7f5c3948a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da165724a0 0x55da165e2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43090/0 (socket says 172.21.15.5:43090) 2024-03-18T15:33:54.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.565+0000 7f5c3948a640 1 -- 172.21.15.5:0/324896243 learned_addr learned my addr 172.21.15.5:0/324896243 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:54.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.565+0000 7f5c38c89640 1 --2- 172.21.15.5:0/324896243 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55da16572dc0 0x55da165e2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:54.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.565+0000 7f5c39c8b640 1 --2- 172.21.15.5:0/324896243 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55da1659d6c0 0x55da165e2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:54.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.565+0000 7f5c3948a640 1 -- 172.21.15.5:0/324896243 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55da1659d6c0 msgr2=0x55da165e2dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:54.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.565+0000 7f5c3948a640 1 --2- 172.21.15.5:0/324896243 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55da1659d6c0 0x55da165e2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:54.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.565+0000 7f5c3948a640 1 -- 172.21.15.5:0/324896243 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55da16572dc0 msgr2=0x55da165e2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:54.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.565+0000 7f5c3948a640 1 --2- 172.21.15.5:0/324896243 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55da16572dc0 0x55da165e2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:54.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.565+0000 7f5c3948a640 1 -- 172.21.15.5:0/324896243 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da1636b070 con 0x55da165724a0 2024-03-18T15:33:54.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.565+0000 7f5c3948a640 1 --2- 172.21.15.5:0/324896243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da165724a0 0x55da165e2350 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5c3001a9f0 tx=0x7f5c300046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b67b9c1a00b6cd9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:54.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.565+0000 7f5c13fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-03-18T15:33:54.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.565+0000 7f5c13fff640 1 -- 172.21.15.5:0/324896243 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c300cb030 con 0x55da165724a0 2024-03-18T15:33:54.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.565+0000 7f5c13fff640 1 -- 172.21.15.5:0/324896243 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f5c30003af0 con 0x55da165724a0 2024-03-18T15:33:54.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.565+0000 7f5c3e870000 1 -- 172.21.15.5:0/324896243 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c2c055d50 con 0x55da165724a0 2024-03-18T15:33:54.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.565+0000 7f5c13fff640 1 -- 172.21.15.5:0/324896243 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f5c300d7050 con 0x55da165724a0 2024-03-18T15:33:54.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.565+0000 7f5c3e870000 10 client.4635 fetch_fsmap learned FSMap version 8 2024-03-18T15:33:54.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.565+0000 7f5c3e870000 1 -- 172.21.15.5:0/324896243 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55da1634f370 con 0x55da165724a0 2024-03-18T15:33:54.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.565+0000 7f5c13fff640 1 -- 172.21.15.5:0/324896243 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c2c055d50 con 0x55da165724a0 2024-03-18T15:33:54.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.565+0000 7f5c13fff640 1 -- 172.21.15.5:0/324896243 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 8) ==== 2284+0+0 (secure 0 0 0) 0x7f5c300da020 con 0x55da165724a0 2024-03-18T15:33:54.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.565+0000 7f5c3e870000 10 client.4635 fetch_fsmap finished waiting for FSMap version 8 2024-03-18T15:33:54.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.565+0000 7f5c3e870000 10 client.4635 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-18T15:33:54.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.565+0000 7f5c3e870000 20 client.4635 populate_metadata read hostname 'smithi005' 2024-03-18T15:33:54.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.565+0000 7f5c3e870000 1 --2- 172.21.15.5:0/324896243 >> [v2:172.21.15.80:6832/1236603350,v1:172.21.15.80:6833/1236603350] conn(0x55da16617670 0x55da16619a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:54.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.565+0000 7f5c3e870000 4 client.4635 mds_command: new command op to 4272 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-18T15:33:54.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.565+0000 7f5c3e870000 1 -- 172.21.15.5:0/324896243 --> [v2:172.21.15.80:6832/1236603350,v1:172.21.15.80:6833/1236603350] -- command(tid 0: {"prefix": "status"}) -- 0x55da163c8a80 con 0x55da16617670 2024-03-18T15:33:54.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.565+0000 7f5c38c89640 1 --2- 172.21.15.5:0/324896243 >> [v2:172.21.15.80:6832/1236603350,v1:172.21.15.80:6833/1236603350] conn(0x55da16617670 0x55da16619a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:54.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.565+0000 7f5c38c89640 1 --2- 172.21.15.5:0/324896243 >> [v2:172.21.15.80:6832/1236603350,v1:172.21.15.80:6833/1236603350] conn(0x55da16617670 0x55da16619a60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=918167b7477fd2e server_cookie=1864c2019c67339b in_seq=0 out_seq=0 2024-03-18T15:33:54.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.565+0000 7f5c13fff640 10 client.4635 ms_handle_connect on v2:172.21.15.80:6832/1236603350 2024-03-18T15:33:54.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.569+0000 7f5c13fff640 1 -- 172.21.15.5:0/324896243 <== mds.0 v2:172.21.15.80:6832/1236603350 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55da163c8a80 con 0x55da16617670 2024-03-18T15:33:54.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.569+0000 7f5c13fff640 10 client.4635 handle_command_reply: tid=0 2024-03-18T15:33:54.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.569+0000 7f5c3e870000 1 client.4635 shutdown 2024-03-18T15:33:54.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.569+0000 7f5c3e870000 1 -- 172.21.15.5:0/324896243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da165724a0 msgr2=0x55da165e2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:54.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.569+0000 7f5c3e870000 1 --2- 172.21.15.5:0/324896243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da165724a0 0x55da165e2350 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5c3001a9f0 tx=0x7f5c300046d0 comp rx=0 tx=0).stop 2024-03-18T15:33:54.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.569+0000 7f5c3e870000 1 -- 172.21.15.5:0/324896243 shutdown_connections 2024-03-18T15:33:54.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.569+0000 7f5c3e870000 1 --2- 172.21.15.5:0/324896243 >> [v2:172.21.15.80:6832/1236603350,v1:172.21.15.80:6833/1236603350] conn(0x55da16617670 0x55da16619a60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:54.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.569+0000 7f5c3e870000 1 --2- 172.21.15.5:0/324896243 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x55da1659d6c0 0x55da165e2dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:54.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.569+0000 7f5c3e870000 1 --2- 172.21.15.5:0/324896243 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55da16572dc0 0x55da165e2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:54.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.569+0000 7f5c13fff640 0 client.4635 ms_handle_reset on v2:172.21.15.80:6832/1236603350 2024-03-18T15:33:54.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.569+0000 7f5c3e870000 1 --2- 172.21.15.5:0/324896243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da165724a0 0x55da165e2350 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:54.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.569+0000 7f5c3e870000 1 -- 172.21.15.5:0/324896243 >> 172.21.15.5:0/324896243 conn(0x55da16598f30 msgr2=0x55da1659b320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:54.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.569+0000 7f5c3e870000 1 -- 172.21.15.5:0/324896243 shutdown_connections 2024-03-18T15:33:54.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.569+0000 7f5c3e870000 1 -- 172.21.15.5:0/324896243 wait complete. 2024-03-18T15:33:54.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.569+0000 7f5c3e870000 20 client.4635 trim_cache size 0 max 16384 2024-03-18T15:33:54.575 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-03-18T15:33:54.576 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "ef54fe03-5aa7-4eb2-9b1e-66c9cc345b94", 2024-03-18T15:33:54.576 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-03-18T15:33:54.576 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 4272, 2024-03-18T15:33:54.576 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-03-18T15:33:54.576 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-03-18T15:33:54.576 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-03-18T15:33:54.576 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 11.578199537, 2024-03-18T15:33:54.576 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 8, 2024-03-18T15:33:54.576 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 21, 2024-03-18T15:33:54.576 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 21, 2024-03-18T15:33:54.576 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 18.570334837000001 2024-03-18T15:33:54.576 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-03-18T15:33:54.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.569+0000 7f5c3b6ee640 1 -- 172.21.15.5:0/2139320552 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5c18067660 msgr2=0x7f5c18069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:54.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.569+0000 7f5c3b6ee640 1 --2- 172.21.15.5:0/2139320552 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5c18067660 0x7f5c18069b20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5c300ad000 tx=0x7f5c300a6000 comp rx=0 tx=0).stop 2024-03-18T15:33:54.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.569+0000 7f5c3b6ee640 1 -- 172.21.15.5:0/2139320552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c3413f6a0 msgr2=0x7f5c341e1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:54.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.569+0000 7f5c3b6ee640 1 --2- 172.21.15.5:0/2139320552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c3413f6a0 0x7f5c341e1790 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c00c730 tx=0x7f5c2c00cbf0 comp rx=0 tx=0).stop 2024-03-18T15:33:54.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.569+0000 7f5c3b6ee640 1 -- 172.21.15.5:0/2139320552 shutdown_connections 2024-03-18T15:33:54.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.569+0000 7f5c3b6ee640 1 --2- 172.21.15.5:0/2139320552 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5c18067660 0x7f5c18069b20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:54.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.569+0000 7f5c3b6ee640 1 --2- 172.21.15.5:0/2139320552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c3413f6a0 0x7f5c341e1790 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:54.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.569+0000 7f5c3b6ee640 1 --2- 172.21.15.5:0/2139320552 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5c3413f260 0x7f5c341e1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:54.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.569+0000 7f5c3b6ee640 1 --2- 172.21.15.5:0/2139320552 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5c3406d2d0 0x7f5c34071aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:54.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.569+0000 7f5c3b6ee640 1 -- 172.21.15.5:0/2139320552 >> 172.21.15.5:0/2139320552 conn(0x7f5c340b5670 msgr2=0x7f5c340b4b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:54.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.569+0000 7f5c3b6ee640 1 -- 172.21.15.5:0/2139320552 shutdown_connections 2024-03-18T15:33:54.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.569+0000 7f5c3b6ee640 1 -- 172.21.15.5:0/2139320552 wait complete. 2024-03-18T15:33:54.587 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-03-18T15:33:54.613 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-03-18T15:33:54.613 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-03-18T15:33:54.613 INFO:tasks.ceph_fuse:client.0 config is {} 2024-03-18T15:33:54.613 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-18T15:33:54.613 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-18T15:33:54.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:33:54.613 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-03-18T15:33:54.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:33:54.619 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link delete ceph-brx 2024-03-18T15:33:54.674 INFO:teuthology.orchestra.run.smithi005.stderr:Cannot find device "ceph-brx" 2024-03-18T15:33:54.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T15:33:54.677 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-03-18T15:33:54.677 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-18T15:33:54.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.837+0000 7fc6142be640 1 Processor -- start 2024-03-18T15:33:54.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.837+0000 7fc6142be640 1 -- start start 2024-03-18T15:33:54.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.837+0000 7fc6142be640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc60c06d2d0 0x7fc60c0b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:54.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.837+0000 7fc6142be640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc60c13f260 0x7fc60c13f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:54.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.837+0000 7fc6142be640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc60c0b0ea0 con 0x7fc60c1453a0 2024-03-18T15:33:54.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.837+0000 7fc6142be640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fc60c0728c0 con 0x7fc60c06d2d0 2024-03-18T15:33:54.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.837+0000 7fc6142be640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fc60c06d6b0 con 0x7fc60c13f260 2024-03-18T15:33:54.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.837+0000 7fc61205a640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc60c06d2d0 0x7fc60c0b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:54.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.837+0000 7fc61205a640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc60c06d2d0 0x7fc60c0b23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43102/0 (socket says 172.21.15.5:43102) 2024-03-18T15:33:54.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.837+0000 7fc61205a640 1 -- 172.21.15.5:0/2398088730 learned_addr learned my addr 172.21.15.5:0/2398088730 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:54.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.837+0000 7fc611859640 1 --2- 172.21.15.5:0/2398088730 >> v2:172.21.15.80:3300/0 conn(0x7fc60c13f260 0x7fc60c13f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:54.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.837+0000 7fc61205a640 1 -- 172.21.15.5:0/2398088730 >> v2:172.21.15.80:3300/0 conn(0x7fc60c13f260 msgr2=0x7fc60c13f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:54.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.837+0000 7fc61205a640 1 --2- 172.21.15.5:0/2398088730 >> v2:172.21.15.80:3300/0 conn(0x7fc60c13f260 0x7fc60c13f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:54.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.837+0000 7fc61205a640 1 -- 172.21.15.5:0/2398088730 >> v1:172.21.15.80:6789/0 conn(0x7fc60c1453a0 legacy=0x7fc60c1457f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:54.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.837+0000 7fc61205a640 1 -- 172.21.15.5:0/2398088730 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc60c0b0bc0 con 0x7fc60c06d2d0 2024-03-18T15:33:54.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.841+0000 7fc61205a640 1 --2- 172.21.15.5:0/2398088730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc60c06d2d0 0x7fc60c0b23a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc60c09eaa0 tx=0x7fc60801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d4e781d8058a52 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:54.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.841+0000 7fc611058640 1 -- 172.21.15.5:0/2398088730 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6080b0070 con 0x7fc60c06d2d0 2024-03-18T15:33:54.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.841+0000 7fc611058640 1 -- 172.21.15.5:0/2398088730 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6080ab600 con 0x7fc60c06d2d0 2024-03-18T15:33:54.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.841+0000 7fc611058640 1 -- 172.21.15.5:0/2398088730 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6080ab9d0 con 0x7fc60c06d2d0 2024-03-18T15:33:54.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.841+0000 7fc6142be640 1 -- 172.21.15.5:0/2398088730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc60c06d2d0 msgr2=0x7fc60c0b23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:54.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.841+0000 7fc6142be640 1 --2- 172.21.15.5:0/2398088730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc60c06d2d0 0x7fc60c0b23a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc60c09eaa0 tx=0x7fc60801c490 comp rx=0 tx=0).stop 2024-03-18T15:33:54.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.845+0000 7fc6142be640 1 -- 172.21.15.5:0/2398088730 shutdown_connections 2024-03-18T15:33:54.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.845+0000 7fc6142be640 1 --2- 172.21.15.5:0/2398088730 >> v2:172.21.15.80:3300/0 conn(0x7fc60c13f260 0x7fc60c13f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:54.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.845+0000 7fc6142be640 1 --2- 172.21.15.5:0/2398088730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc60c06d2d0 0x7fc60c0b23a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:54.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.845+0000 7fc6142be640 1 -- 172.21.15.5:0/2398088730 >> 172.21.15.5:0/2398088730 conn(0x7fc60c0b5670 msgr2=0x7fc60c0b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:54.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.845+0000 7fc6142be640 1 -- 172.21.15.5:0/2398088730 shutdown_connections 2024-03-18T15:33:54.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.845+0000 7fc6142be640 1 -- 172.21.15.5:0/2398088730 wait complete. 2024-03-18T15:33:54.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.845+0000 7fc6142be640 1 Processor -- start 2024-03-18T15:33:54.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.845+0000 7fc6142be640 1 -- start start 2024-03-18T15:33:54.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.845+0000 7fc6142be640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc60c06d2d0 0x7fc60c13dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:54.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.845+0000 7fc6142be640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc60c13f260 0x7fc60c0af5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:54.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.845+0000 7fc6142be640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc60c1453a0 0x7fc60c135760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:54.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.845+0000 7fc6142be640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc60c072ed0 con 0x7fc60c06d2d0 2024-03-18T15:33:54.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.845+0000 7fc6142be640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc60c072d50 con 0x7fc60c13f260 2024-03-18T15:33:54.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.845+0000 7fc6142be640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fc60c0b5140 con 0x7fc60c1453a0 2024-03-18T15:33:54.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.845+0000 7fc61205a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc60c06d2d0 0x7fc60c13dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:54.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.845+0000 7fc61205a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc60c06d2d0 0x7fc60c13dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43108/0 (socket says 172.21.15.5:43108) 2024-03-18T15:33:54.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.845+0000 7fc61205a640 1 -- 172.21.15.5:0/4188229009 learned_addr learned my addr 172.21.15.5:0/4188229009 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:54.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.845+0000 7fc611859640 1 --2- 172.21.15.5:0/4188229009 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc60c13f260 0x7fc60c0af5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:54.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.845+0000 7fc61285b640 1 --2- 172.21.15.5:0/4188229009 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc60c1453a0 0x7fc60c135760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:54.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.845+0000 7fc61205a640 1 -- 172.21.15.5:0/4188229009 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc60c1453a0 msgr2=0x7fc60c135760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:54.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.845+0000 7fc61205a640 1 --2- 172.21.15.5:0/4188229009 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc60c1453a0 0x7fc60c135760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:54.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.845+0000 7fc61205a640 1 -- 172.21.15.5:0/4188229009 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc60c13f260 msgr2=0x7fc60c0af5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:54.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.845+0000 7fc61205a640 1 --2- 172.21.15.5:0/4188229009 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc60c13f260 0x7fc60c0af5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:54.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.845+0000 7fc61205a640 1 -- 172.21.15.5:0/4188229009 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc60c136e90 con 0x7fc60c06d2d0 2024-03-18T15:33:54.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.845+0000 7fc611859640 1 --2- 172.21.15.5:0/4188229009 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc60c13f260 0x7fc60c0af5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:33:54.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.845+0000 7fc61205a640 1 --2- 172.21.15.5:0/4188229009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc60c06d2d0 0x7fc60c13dcb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc6080ae040 tx=0x7fc608019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:54.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.845+0000 7fc602ffd640 1 -- 172.21.15.5:0/4188229009 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6080b0040 con 0x7fc60c06d2d0 2024-03-18T15:33:54.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.845+0000 7fc602ffd640 1 -- 172.21.15.5:0/4188229009 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6080b4430 con 0x7fc60c06d2d0 2024-03-18T15:33:54.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.845+0000 7fc6142be640 1 -- 172.21.15.5:0/4188229009 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc60c0aefc0 con 0x7fc60c06d2d0 2024-03-18T15:33:54.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.845+0000 7fc602ffd640 1 -- 172.21.15.5:0/4188229009 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6080b4840 con 0x7fc60c06d2d0 2024-03-18T15:33:54.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.845+0000 7fc6142be640 1 -- 172.21.15.5:0/4188229009 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc60c1352d0 con 0x7fc60c06d2d0 2024-03-18T15:33:54.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.845+0000 7fc6142be640 1 -- 172.21.15.5:0/4188229009 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc60c04f550 con 0x7fc60c06d2d0 2024-03-18T15:33:54.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.849+0000 7fc602ffd640 1 -- 172.21.15.5:0/4188229009 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fc6080b49e0 con 0x7fc60c06d2d0 2024-03-18T15:33:54.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.849+0000 7fc602ffd640 1 --2- 172.21.15.5:0/4188229009 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fc5f0067660 0x7fc5f0069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:54.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.849+0000 7fc602ffd640 1 -- 172.21.15.5:0/4188229009 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc60811f240 con 0x7fc60c06d2d0 2024-03-18T15:33:54.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.849+0000 7fc611859640 1 --2- 172.21.15.5:0/4188229009 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fc5f0067660 0x7fc5f0069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:54.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.849+0000 7fc611859640 1 --2- 172.21.15.5:0/4188229009 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fc5f0067660 0x7fc5f0069b20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc60c14fe90 tx=0x7fc5fc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:54.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:54.853+0000 7fc602ffd640 1 -- 172.21.15.5:0/4188229009 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fc608016110 con 0x7fc60c06d2d0 2024-03-18T15:33:55.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:55.053+0000 7fc6142be640 1 -- 172.21.15.5:0/4188229009 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc60c13d8d0 con 0x7fc60c06d2d0 2024-03-18T15:33:55.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:55.053+0000 7fc602ffd640 1 -- 172.21.15.5:0/4188229009 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fc6080eff50 con 0x7fc60c06d2d0 2024-03-18T15:33:55.054 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-18T15:33:55.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:55.053+0000 7fc6142be640 1 -- 172.21.15.5:0/4188229009 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fc5f0067660 msgr2=0x7fc5f0069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:55.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:55.053+0000 7fc6142be640 1 --2- 172.21.15.5:0/4188229009 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fc5f0067660 0x7fc5f0069b20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc60c14fe90 tx=0x7fc5fc008040 comp rx=0 tx=0).stop 2024-03-18T15:33:55.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:55.053+0000 7fc6142be640 1 -- 172.21.15.5:0/4188229009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc60c06d2d0 msgr2=0x7fc60c13dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:55.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:55.053+0000 7fc6142be640 1 --2- 172.21.15.5:0/4188229009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc60c06d2d0 0x7fc60c13dcb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc6080ae040 tx=0x7fc608019230 comp rx=0 tx=0).stop 2024-03-18T15:33:55.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:55.053+0000 7fc6142be640 1 -- 172.21.15.5:0/4188229009 shutdown_connections 2024-03-18T15:33:55.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:55.053+0000 7fc6142be640 1 --2- 172.21.15.5:0/4188229009 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fc5f0067660 0x7fc5f0069b20 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:55.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:55.053+0000 7fc6142be640 1 --2- 172.21.15.5:0/4188229009 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc60c1453a0 0x7fc60c135760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:55.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:55.053+0000 7fc6142be640 1 --2- 172.21.15.5:0/4188229009 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc60c13f260 0x7fc60c0af5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:55.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:55.053+0000 7fc6142be640 1 --2- 172.21.15.5:0/4188229009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc60c06d2d0 0x7fc60c13dcb0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:55.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:55.053+0000 7fc6142be640 1 -- 172.21.15.5:0/4188229009 >> 172.21.15.5:0/4188229009 conn(0x7fc60c0b5670 msgr2=0x7fc60c06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:55.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:55.053+0000 7fc6142be640 1 -- 172.21.15.5:0/4188229009 shutdown_connections 2024-03-18T15:33:55.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:55.053+0000 7fc6142be640 1 -- 172.21.15.5:0/4188229009 wait complete. 2024-03-18T15:33:55.073 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-18T15:33:55.073 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-18T15:33:55.073 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-03-18T15:33:55.073 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-18T15:33:55.073 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-18T15:33:55.073 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-18T15:33:55.073 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-18T15:33:55.074 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-18T15:33:55.074 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-18T15:33:55.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:33:55.074 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-03-18T15:33:55.080 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-18T15:33:55.080 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-18T15:33:55.080 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-03-18T15:33:55.081 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-03-18T15:33:55.081 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-03-18T15:33:55.081 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-03-18T15:33:55.081 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-18T15:33:55.081 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-03-18T15:33:55.081 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-03-18T15:33:55.081 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-18T15:33:55.081 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-03-18T15:33:55.081 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-03-18T15:33:55.081 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-18T15:33:55.082 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-03-18T15:33:55.082 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-03-18T15:33:55.082 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-03-18T15:33:55.082 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42462sec preferred_lft 42462sec 2024-03-18T15:33:55.082 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link 2024-03-18T15:33:55.082 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-03-18T15:33:55.082 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-18T15:33:55.082 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-03-18T15:33:55.082 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-03-18T15:33:55.082 INFO:teuthology.orchestra.run.smithi005.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-18T15:33:55.083 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 02:42:57:05:8c:91 brd ff:ff:ff:ff:ff:ff 2024-03-18T15:33:55.083 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-18T15:33:55.083 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-03-18T15:33:55.085 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-18T15:33:55.085 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-18T15:33:55.086 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-03-18T15:33:55.086 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link add name ceph-brx type bridge 2024-03-18T15:33:55.086 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip addr flush dev ceph-brx 2024-03-18T15:33:55.086 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set ceph-brx up 2024-03-18T15:33:55.086 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-18T15:33:55.086 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-03-18T15:33:55.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-18T15:33:55.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T15:33:55.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:33:55.211 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-18T15:33:55.222 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-03-18T15:33:55.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:33:55.224 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-03-18T15:33:55.293 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-03-18T15:33:55.293 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-18T15:33:55.293 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-03-18T15:33:55.293 INFO:teuthology.orchestra.run.smithi005.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-18T15:33:55.293 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-03-18T15:33:55.293 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-18T15:33:55.294 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-18T15:33:55.294 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-03-18T15:33:55.294 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-18T15:33:55.294 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-18T15:33:55.294 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-18T15:33:55.294 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-03-18T15:33:55.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-18T15:33:55.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T15:33:55.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:33:55.426 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-03-18T15:33:55.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:33:55.433 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list-id 2024-03-18T15:33:55.478 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-18T15:33:55.479 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-03-18T15:33:55.479 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-18T15:33:55.479 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-18T15:33:55.479 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-03-18T15:33:55.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-18T15:33:55.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T15:33:55.604 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-18T15:33:55.604 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-18T15:33:55.604 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-03-18T15:33:55.604 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-18T15:33:55.604 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-18T15:33:55.604 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-18T15:33:55.605 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-18T15:33:55.605 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-18T15:33:55.605 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-03-18T15:33:55.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-18T15:33:55.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T15:33:55.717 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-18T15:33:55.717 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-03-18T15:33:55.717 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-03-18T15:33:55.717 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set dev brx.0 master ceph-brx 2024-03-18T15:33:55.717 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-03-18T15:33:55.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-18T15:33:55.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T15:33:55.775 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-18T15:33:55.776 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-18T15:33:55.776 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-18T15:33:55.821 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-18T15:33:55.821 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-18T15:33:55.822 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-18T15:33:55.870 DEBUG:teuthology.orchestra.run.smithi005:> sudo modprobe fuse 2024-03-18T15:33:55.927 DEBUG:teuthology.orchestra.run.smithi005:> cat /proc/self/mounts | awk '{print $2}' 2024-03-18T15:33:55.975 INFO:teuthology.orchestra.run.smithi005.stdout:/sys 2024-03-18T15:33:55.975 INFO:teuthology.orchestra.run.smithi005.stdout:/proc 2024-03-18T15:33:55.975 INFO:teuthology.orchestra.run.smithi005.stdout:/dev 2024-03-18T15:33:55.975 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/pts 2024-03-18T15:33:55.975 INFO:teuthology.orchestra.run.smithi005.stdout:/run 2024-03-18T15:33:55.975 INFO:teuthology.orchestra.run.smithi005.stdout:/ 2024-03-18T15:33:55.975 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/security 2024-03-18T15:33:55.975 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/shm 2024-03-18T15:33:55.976 INFO:teuthology.orchestra.run.smithi005.stdout:/run/lock 2024-03-18T15:33:55.976 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup 2024-03-18T15:33:55.976 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/pstore 2024-03-18T15:33:55.976 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/bpf 2024-03-18T15:33:55.976 INFO:teuthology.orchestra.run.smithi005.stdout:/proc/sys/fs/binfmt_misc 2024-03-18T15:33:55.976 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/hugepages 2024-03-18T15:33:55.976 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/mqueue 2024-03-18T15:33:55.976 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/debug 2024-03-18T15:33:55.976 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/tracing 2024-03-18T15:33:55.976 INFO:teuthology.orchestra.run.smithi005.stdout:/proc/fs/nfsd 2024-03-18T15:33:55.976 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/fuse/connections 2024-03-18T15:33:55.976 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/config 2024-03-18T15:33:55.976 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-sysusers.service 2024-03-18T15:33:55.976 INFO:teuthology.orchestra.run.smithi005.stdout:/run/rpc_pipefs 2024-03-18T15:33:55.977 INFO:teuthology.orchestra.run.smithi005.stdout:/run/user/1000 2024-03-18T15:33:55.977 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph 2024-03-18T15:33:55.977 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-0 2024-03-18T15:33:55.977 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-1 2024-03-18T15:33:55.977 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-2 2024-03-18T15:33:55.977 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-3 2024-03-18T15:33:55.977 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns 2024-03-18T15:33:55.977 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-18T15:33:55.977 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-18T15:33:55.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:33:55.978 DEBUG:teuthology.orchestra.run.smithi005:> ls /sys/fs/fuse/connections 2024-03-18T15:33:56.026 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-18T15:33:56.026 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-18T15:33:56.075 DEBUG:teuthology.orchestra.run.smithi005:> sudo modprobe fuse 2024-03-18T15:33:56.089 DEBUG:teuthology.orchestra.run.smithi005:> cat /proc/self/mounts | awk '{print $2}' 2024-03-18T15:33:56.138 INFO:teuthology.orchestra.run.smithi005.stdout:/sys 2024-03-18T15:33:56.138 INFO:teuthology.orchestra.run.smithi005.stdout:/proc 2024-03-18T15:33:56.138 INFO:teuthology.orchestra.run.smithi005.stdout:/dev 2024-03-18T15:33:56.138 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/pts 2024-03-18T15:33:56.138 INFO:teuthology.orchestra.run.smithi005.stdout:/run 2024-03-18T15:33:56.138 INFO:teuthology.orchestra.run.smithi005.stdout:/ 2024-03-18T15:33:56.138 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/security 2024-03-18T15:33:56.138 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/shm 2024-03-18T15:33:56.139 INFO:teuthology.orchestra.run.smithi005.stdout:/run/lock 2024-03-18T15:33:56.139 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup 2024-03-18T15:33:56.139 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/pstore 2024-03-18T15:33:56.139 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/bpf 2024-03-18T15:33:56.139 INFO:teuthology.orchestra.run.smithi005.stdout:/proc/sys/fs/binfmt_misc 2024-03-18T15:33:56.139 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/hugepages 2024-03-18T15:33:56.139 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/mqueue 2024-03-18T15:33:56.139 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/debug 2024-03-18T15:33:56.139 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/tracing 2024-03-18T15:33:56.139 INFO:teuthology.orchestra.run.smithi005.stdout:/proc/fs/nfsd 2024-03-18T15:33:56.139 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/fuse/connections 2024-03-18T15:33:56.139 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/config 2024-03-18T15:33:56.139 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-sysusers.service 2024-03-18T15:33:56.139 INFO:teuthology.orchestra.run.smithi005.stdout:/run/rpc_pipefs 2024-03-18T15:33:56.140 INFO:teuthology.orchestra.run.smithi005.stdout:/run/user/1000 2024-03-18T15:33:56.140 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph 2024-03-18T15:33:56.140 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-0 2024-03-18T15:33:56.140 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-1 2024-03-18T15:33:56.140 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-2 2024-03-18T15:33:56.140 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-3 2024-03-18T15:33:56.140 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns 2024-03-18T15:33:56.140 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-18T15:33:56.140 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-18T15:33:56.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:33:56.141 DEBUG:teuthology.orchestra.run.smithi005:> ls /sys/fs/fuse/connections 2024-03-18T15:33:56.175 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi005.stderr:2024-03-18T15:33:56.173+0000 7fe46144b580 -1 init, newargv = 0x559c719eadb0 newargc=15 2024-03-18T15:33:56.176 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi005.stderr:ceph-fuse[33062]: starting ceph client 2024-03-18T15:33:56.182 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi005.stderr:ceph-fuse[33062]: starting fuse 2024-03-18T15:33:57.281 DEBUG:teuthology.orchestra.run.smithi005:> sudo modprobe fuse 2024-03-18T15:33:57.298 DEBUG:teuthology.orchestra.run.smithi005:> cat /proc/self/mounts | awk '{print $2}' 2024-03-18T15:33:57.347 INFO:teuthology.orchestra.run.smithi005.stdout:/sys 2024-03-18T15:33:57.347 INFO:teuthology.orchestra.run.smithi005.stdout:/proc 2024-03-18T15:33:57.347 INFO:teuthology.orchestra.run.smithi005.stdout:/dev 2024-03-18T15:33:57.347 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/pts 2024-03-18T15:33:57.347 INFO:teuthology.orchestra.run.smithi005.stdout:/run 2024-03-18T15:33:57.347 INFO:teuthology.orchestra.run.smithi005.stdout:/ 2024-03-18T15:33:57.347 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/security 2024-03-18T15:33:57.347 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/shm 2024-03-18T15:33:57.347 INFO:teuthology.orchestra.run.smithi005.stdout:/run/lock 2024-03-18T15:33:57.347 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup 2024-03-18T15:33:57.347 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/pstore 2024-03-18T15:33:57.348 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/bpf 2024-03-18T15:33:57.348 INFO:teuthology.orchestra.run.smithi005.stdout:/proc/sys/fs/binfmt_misc 2024-03-18T15:33:57.348 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/hugepages 2024-03-18T15:33:57.348 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/mqueue 2024-03-18T15:33:57.348 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/debug 2024-03-18T15:33:57.348 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/tracing 2024-03-18T15:33:57.348 INFO:teuthology.orchestra.run.smithi005.stdout:/proc/fs/nfsd 2024-03-18T15:33:57.348 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/fuse/connections 2024-03-18T15:33:57.348 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/config 2024-03-18T15:33:57.348 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-sysusers.service 2024-03-18T15:33:57.348 INFO:teuthology.orchestra.run.smithi005.stdout:/run/rpc_pipefs 2024-03-18T15:33:57.348 INFO:teuthology.orchestra.run.smithi005.stdout:/run/user/1000 2024-03-18T15:33:57.348 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph 2024-03-18T15:33:57.348 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-0 2024-03-18T15:33:57.349 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-1 2024-03-18T15:33:57.349 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-2 2024-03-18T15:33:57.349 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-3 2024-03-18T15:33:57.349 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns 2024-03-18T15:33:57.349 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-18T15:33:57.349 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-18T15:33:57.349 INFO:teuthology.orchestra.run.smithi005.stdout:/home/ubuntu/cephtest/mnt.0 2024-03-18T15:33:57.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:33:57.350 DEBUG:teuthology.orchestra.run.smithi005:> ls /sys/fs/fuse/connections 2024-03-18T15:33:57.398 INFO:teuthology.orchestra.run.smithi005.stdout:45 2024-03-18T15:33:57.399 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-03-18T15:33:57.399 DEBUG:teuthology.orchestra.run.smithi005:> sudo stdin-killer -- python3 -c ' 2024-03-18T15:33:57.399 DEBUG:teuthology.orchestra.run.smithi005:> import glob 2024-03-18T15:33:57.399 DEBUG:teuthology.orchestra.run.smithi005:> import re 2024-03-18T15:33:57.399 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-03-18T15:33:57.399 DEBUG:teuthology.orchestra.run.smithi005:> import subprocess 2024-03-18T15:33:57.399 DEBUG:teuthology.orchestra.run.smithi005:> 2024-03-18T15:33:57.399 DEBUG:teuthology.orchestra.run.smithi005:> def _find_admin_socket(client_name): 2024-03-18T15:33:57.399 DEBUG:teuthology.orchestra.run.smithi005:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-18T15:33:57.399 DEBUG:teuthology.orchestra.run.smithi005:> files = glob.glob(asok_path) 2024-03-18T15:33:57.399 DEBUG:teuthology.orchestra.run.smithi005:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-18T15:33:57.400 DEBUG:teuthology.orchestra.run.smithi005:> 2024-03-18T15:33:57.400 DEBUG:teuthology.orchestra.run.smithi005:> # Given a non-glob path, it better be there 2024-03-18T15:33:57.400 DEBUG:teuthology.orchestra.run.smithi005:> if "*" not in asok_path: 2024-03-18T15:33:57.400 DEBUG:teuthology.orchestra.run.smithi005:> assert(len(files) == 1) 2024-03-18T15:33:57.400 DEBUG:teuthology.orchestra.run.smithi005:> return files[0] 2024-03-18T15:33:57.400 DEBUG:teuthology.orchestra.run.smithi005:> 2024-03-18T15:33:57.400 DEBUG:teuthology.orchestra.run.smithi005:> for f in files: 2024-03-18T15:33:57.400 DEBUG:teuthology.orchestra.run.smithi005:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-18T15:33:57.400 DEBUG:teuthology.orchestra.run.smithi005:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-18T15:33:57.400 DEBUG:teuthology.orchestra.run.smithi005:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-18T15:33:57.400 DEBUG:teuthology.orchestra.run.smithi005:> contents = proc_f.read() 2024-03-18T15:33:57.400 DEBUG:teuthology.orchestra.run.smithi005:> if mountpoint in contents: 2024-03-18T15:33:57.400 DEBUG:teuthology.orchestra.run.smithi005:> return f 2024-03-18T15:33:57.400 DEBUG:teuthology.orchestra.run.smithi005:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-18T15:33:57.400 DEBUG:teuthology.orchestra.run.smithi005:> 2024-03-18T15:33:57.400 DEBUG:teuthology.orchestra.run.smithi005:> print(_find_admin_socket("client.0")) 2024-03-18T15:33:57.401 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-03-18T15:33:57.525 INFO:teuthology.orchestra.run.smithi005.stdout:/var/run/ceph/ceph-client.0.33062.asok 2024-03-18T15:33:57.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T15:33:57.538 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.33062.asok 2024-03-18T15:33:57.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:33:57.539 DEBUG:teuthology.orchestra.run.smithi005:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.33062.asok status 2024-03-18T15:33:57.698 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-03-18T15:33:57.698 INFO:teuthology.orchestra.run.smithi005.stdout: "metadata": { 2024-03-18T15:33:57.698 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_sha1": "3d78f5ba07bf18880172c07d4d7b4ac00c9d30af", 2024-03-18T15:33:57.699 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_version": "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)", 2024-03-18T15:33:57.699 INFO:teuthology.orchestra.run.smithi005.stdout: "entity_id": "0", 2024-03-18T15:33:57.699 INFO:teuthology.orchestra.run.smithi005.stdout: "hostname": "smithi005", 2024-03-18T15:33:57.699 INFO:teuthology.orchestra.run.smithi005.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-18T15:33:57.699 INFO:teuthology.orchestra.run.smithi005.stdout: "pid": "33062", 2024-03-18T15:33:57.699 INFO:teuthology.orchestra.run.smithi005.stdout: "root": "/" 2024-03-18T15:33:57.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-03-18T15:33:57.699 INFO:teuthology.orchestra.run.smithi005.stdout: "dentry_count": 0, 2024-03-18T15:33:57.699 INFO:teuthology.orchestra.run.smithi005.stdout: "dentry_pinned_count": 0, 2024-03-18T15:33:57.699 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 4650, 2024-03-18T15:33:57.699 INFO:teuthology.orchestra.run.smithi005.stdout: "inst": { 2024-03-18T15:33:57.699 INFO:teuthology.orchestra.run.smithi005.stdout: "name": { 2024-03-18T15:33:57.699 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "client", 2024-03-18T15:33:57.699 INFO:teuthology.orchestra.run.smithi005.stdout: "num": 4650 2024-03-18T15:33:57.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-03-18T15:33:57.700 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": { 2024-03-18T15:33:57.700 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-03-18T15:33:57.700 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "192.168.0.1:0", 2024-03-18T15:33:57.700 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1470576524 2024-03-18T15:33:57.700 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-03-18T15:33:57.700 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-03-18T15:33:57.700 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": { 2024-03-18T15:33:57.700 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-03-18T15:33:57.700 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "192.168.0.1:0", 2024-03-18T15:33:57.700 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1470576524 2024-03-18T15:33:57.700 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-03-18T15:33:57.700 INFO:teuthology.orchestra.run.smithi005.stdout: "inst_str": "client.4650 192.168.0.1:0/1470576524", 2024-03-18T15:33:57.700 INFO:teuthology.orchestra.run.smithi005.stdout: "addr_str": "192.168.0.1:0/1470576524", 2024-03-18T15:33:57.700 INFO:teuthology.orchestra.run.smithi005.stdout: "inode_count": 1, 2024-03-18T15:33:57.701 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_epoch": 8, 2024-03-18T15:33:57.701 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_epoch": 21, 2024-03-18T15:33:57.701 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_epoch_barrier": 0, 2024-03-18T15:33:57.701 INFO:teuthology.orchestra.run.smithi005.stdout: "blocklisted": false, 2024-03-18T15:33:57.701 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs" 2024-03-18T15:33:57.701 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-03-18T15:33:57.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:33:57.714 DEBUG:teuthology.orchestra.run.smithi005:> stat --file-system '--printf=%T 2024-03-18T15:33:57.714 DEBUG:teuthology.orchestra.run.smithi005:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-18T15:33:57.720 INFO:teuthology.orchestra.run.smithi005.stdout:fuseblk 2024-03-18T15:33:57.720 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-18T15:33:57.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:33:57.721 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-18T15:33:57.773 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-03-18T15:33:57.790 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:33:57.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.945+0000 7fadbe076640 1 Processor -- start 2024-03-18T15:33:57.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.945+0000 7fadbe076640 1 -- start start 2024-03-18T15:33:57.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.945+0000 7fadbe076640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fadb813f6a0 0x7fadb81513a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:57.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.945+0000 7fadbe076640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadb80b2ab0 con 0x7fadb806d2d0 2024-03-18T15:33:57.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.945+0000 7fadbe076640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadb80b0ea0 con 0x7fadb813f260 2024-03-18T15:33:57.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.945+0000 7fadbe076640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fadb80728c0 con 0x7fadb813f6a0 2024-03-18T15:33:57.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.945+0000 7fadb7fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fadb813f6a0 0x7fadb81513a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:57.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.945+0000 7fadb7fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fadb813f6a0 0x7fadb81513a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43120/0 (socket says 172.21.15.5:43120) 2024-03-18T15:33:57.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.945+0000 7fadb7fff640 1 -- 172.21.15.5:0/148987201 learned_addr learned my addr 172.21.15.5:0/148987201 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:57.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.945+0000 7fadb77fe640 1 --1- 172.21.15.5:0/148987201 >> v1:172.21.15.5:6789/0 conn(0x7fadb806d2d0 0x7fadb80b23a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46932/0 (socket says 172.21.15.5:46932) 2024-03-18T15:33:57.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.945+0000 7fadb7fff640 1 -- 172.21.15.5:0/148987201 >> v1:172.21.15.80:6789/0 conn(0x7fadb813f260 legacy=0x7fadb814dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:57.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.945+0000 7fadb7fff640 1 -- 172.21.15.5:0/148987201 >> v1:172.21.15.5:6789/0 conn(0x7fadb806d2d0 legacy=0x7fadb80b23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:57.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.945+0000 7fadb67fc640 1 -- 172.21.15.5:0/148987201 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3139924199 0 0) 0x7fadb80b2ab0 con 0x7fadb806d2d0 2024-03-18T15:33:57.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.945+0000 7fadb7fff640 1 -- 172.21.15.5:0/148987201 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadb806d6b0 con 0x7fadb813f6a0 2024-03-18T15:33:57.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.949+0000 7fadb7fff640 1 --2- 172.21.15.5:0/148987201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadb813f6a0 0x7fadb81513a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fadb0009e50 tx=0x7fadb001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d7cd2cac616411 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:57.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.949+0000 7fadb67fc640 1 -- 172.21.15.5:0/148987201 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb00a5860 con 0x7fadb813f6a0 2024-03-18T15:33:57.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.949+0000 7fadb67fc640 1 -- 172.21.15.5:0/148987201 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadb00a5a00 con 0x7fadb813f6a0 2024-03-18T15:33:57.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.949+0000 7fadb67fc640 1 -- 172.21.15.5:0/148987201 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb00a5dd0 con 0x7fadb813f6a0 2024-03-18T15:33:57.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.949+0000 7fadbe076640 1 -- 172.21.15.5:0/148987201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadb813f6a0 msgr2=0x7fadb81513a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:57.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.949+0000 7fadbe076640 1 --2- 172.21.15.5:0/148987201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadb813f6a0 0x7fadb81513a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fadb0009e50 tx=0x7fadb001c9a0 comp rx=0 tx=0).stop 2024-03-18T15:33:57.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.949+0000 7fadbe076640 1 -- 172.21.15.5:0/148987201 shutdown_connections 2024-03-18T15:33:57.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.949+0000 7fadbe076640 1 --2- 172.21.15.5:0/148987201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadb813f6a0 0x7fadb81513a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:57.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.949+0000 7fadbe076640 1 -- 172.21.15.5:0/148987201 >> 172.21.15.5:0/148987201 conn(0x7fadb80b5670 msgr2=0x7fadb80b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:57.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.949+0000 7fadbe076640 1 -- 172.21.15.5:0/148987201 shutdown_connections 2024-03-18T15:33:57.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.949+0000 7fadbe076640 1 -- 172.21.15.5:0/148987201 wait complete. 2024-03-18T15:33:57.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.949+0000 7fadbe076640 1 Processor -- start 2024-03-18T15:33:57.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.949+0000 7fadbe076640 1 -- start start 2024-03-18T15:33:57.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.949+0000 7fadbe076640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fadb806d2d0 0x7fadb8145ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:57.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.949+0000 7fadbe076640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadb813f260 0x7fadb81408f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:57.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.949+0000 7fadbe076640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fadb813f6a0 0x7fadb8140e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:57.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.949+0000 7fadbe076640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fadb80b01f0 con 0x7fadb813f260 2024-03-18T15:33:57.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.949+0000 7fadbe076640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fadb80b0070 con 0x7fadb806d2d0 2024-03-18T15:33:57.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.949+0000 7fadbe076640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fadb80b07e0 con 0x7fadb813f6a0 2024-03-18T15:33:57.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.949+0000 7fadb6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadb813f260 0x7fadb81408f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:57.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.949+0000 7fadb6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadb813f260 0x7fadb81408f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43124/0 (socket says 172.21.15.5:43124) 2024-03-18T15:33:57.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.949+0000 7fadb6ffd640 1 -- 172.21.15.5:0/3051589515 learned_addr learned my addr 172.21.15.5:0/3051589515 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:57.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.949+0000 7fadb77fe640 1 --2- 172.21.15.5:0/3051589515 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fadb806d2d0 0x7fadb8145ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:57.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.949+0000 7fadb7fff640 1 --2- 172.21.15.5:0/3051589515 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fadb813f6a0 0x7fadb8140e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:57.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.949+0000 7fadb6ffd640 1 -- 172.21.15.5:0/3051589515 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fadb813f6a0 msgr2=0x7fadb8140e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:57.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.949+0000 7fadb6ffd640 1 --2- 172.21.15.5:0/3051589515 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fadb813f6a0 0x7fadb8140e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:57.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.949+0000 7fadb6ffd640 1 -- 172.21.15.5:0/3051589515 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fadb806d2d0 msgr2=0x7fadb8145ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:57.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.949+0000 7fadb6ffd640 1 --2- 172.21.15.5:0/3051589515 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fadb806d2d0 0x7fadb8145ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:57.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.949+0000 7fadb6ffd640 1 -- 172.21.15.5:0/3051589515 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadb8136bf0 con 0x7fadb813f260 2024-03-18T15:33:57.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.949+0000 7fadb6ffd640 1 --2- 172.21.15.5:0/3051589515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadb813f260 0x7fadb81408f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fada800c500 tx=0x7fada800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:57.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.949+0000 7fad9ffff640 1 -- 172.21.15.5:0/3051589515 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada8016020 con 0x7fadb813f260 2024-03-18T15:33:57.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.949+0000 7fad9ffff640 1 -- 172.21.15.5:0/3051589515 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fada8005220 con 0x7fadb813f260 2024-03-18T15:33:57.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.949+0000 7fadbe076640 1 -- 172.21.15.5:0/3051589515 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadb813feb0 con 0x7fadb813f260 2024-03-18T15:33:57.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.949+0000 7fad9ffff640 1 -- 172.21.15.5:0/3051589515 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada8007030 con 0x7fadb813f260 2024-03-18T15:33:57.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.949+0000 7fadbe076640 1 -- 172.21.15.5:0/3051589515 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadb8144df0 con 0x7fadb813f260 2024-03-18T15:33:57.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.953+0000 7fadbe076640 1 -- 172.21.15.5:0/3051589515 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadb804f550 con 0x7fadb813f260 2024-03-18T15:33:57.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.953+0000 7fad9ffff640 1 -- 172.21.15.5:0/3051589515 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fada80071f0 con 0x7fadb813f260 2024-03-18T15:33:57.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.953+0000 7fad9ffff640 1 --2- 172.21.15.5:0/3051589515 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fada4067660 0x7fada4069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:57.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.953+0000 7fadb77fe640 1 --2- 172.21.15.5:0/3051589515 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fada4067660 0x7fada4069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:57.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.953+0000 7fad9ffff640 1 -- 172.21.15.5:0/3051589515 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fada8085240 con 0x7fadb813f260 2024-03-18T15:33:57.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.953+0000 7fadb77fe640 1 --2- 172.21.15.5:0/3051589515 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fada4067660 0x7fada4069b20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fadac007fd0 tx=0x7fadac007ba0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:57.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:57.957+0000 7fad9ffff640 1 -- 172.21.15.5:0/3051589515 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fada8020020 con 0x7fadb813f260 2024-03-18T15:33:58.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.125+0000 7fadbe076640 1 -- 172.21.15.5:0/3051589515 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fadb813daa0 con 0x7fadb813f260 2024-03-18T15:33:58.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.125+0000 7fad9ffff640 1 -- 172.21.15.5:0/3051589515 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16449 (secure 0 0 0) 0x7fada8055ed0 con 0x7fadb813f260 2024-03-18T15:33:58.129 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:33:58.129 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":21,"fsid":"ef54fe03-5aa7-4eb2-9b1e-66c9cc345b94","created":"2024-03-18T15:33:18.086614+0000","modified":"2024-03-18T15:33:43.984050+0000","last_up_change":"2024-03-18T15:33:29.229911+0000","last_in_change":"2024-03-18T15:33:20.323840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:28.963186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T15:33:40.533646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T15:33:41.323519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df1c754b-af59-4d16-8919-fbb6386b3c48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6804","nonce":976264754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6808","nonce":976264754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6815","nonce":976264754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6811","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6813","nonce":976264754}]},"public_addr":"172.21.15.5:6804/976264754","cluster_addr":"172.21.15.5:6808/976264754","heartbeat_back_addr":"172.21.15.5:6815/976264754","heartbeat_front_addr":"172.21.15.5:6813/976264754","state":["exists","up"]},{"osd":1,"uuid":"b9a08b5e-8af4-4144-bdba-d69ed1ac627a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6801","nonce":3901761126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6803","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6805","nonce":3901761126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6812","nonce":3901761126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6807","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6809","nonce":3901761126}]},"public_addr":"172.21.15.5:6801/3901761126","cluster_addr":"172.21.15.5:6805/3901761126","heartbeat_back_addr":"172.21.15.5:6812/3901761126","heartbeat_front_addr":"172.21.15.5:6809/3901761126","state":["exists","up"]},{"osd":2,"uuid":"c3e8bb76-e382-439d-b380-62ef92f1255a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6817","nonce":502968448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6819","nonce":502968448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6823","nonce":502968448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6821","nonce":502968448}]},"public_addr":"172.21.15.5:6817/502968448","cluster_addr":"172.21.15.5:6819/502968448","heartbeat_back_addr":"172.21.15.5:6823/502968448","heartbeat_front_addr":"172.21.15.5:6821/502968448","state":["exists","up"]},{"osd":3,"uuid":"e8c94625-f2cd-4eba-b53d-2e7356e1c02a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6825","nonce":2425932735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6827","nonce":2425932735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6831","nonce":2425932735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6829","nonce":2425932735}]},"public_addr":"172.21.15.5:6825/2425932735","cluster_addr":"172.21.15.5:6827/2425932735","heartbeat_back_addr":"172.21.15.5:6831/2425932735","heartbeat_front_addr":"172.21.15.5:6829/2425932735","state":["exists","up"]},{"osd":4,"uuid":"0099dd52-6265-4d32-82e4-3ef16e73add0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6817","nonce":976514285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6819","nonce":976514285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6823","nonce":976514285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6821","nonce":976514285}]},"public_addr":"172.21.15.80:6817/976514285","cluster_addr":"172.21.15.80:6819/976514285","heartbeat_back_addr":"172.21.15.80:6823/976514285","heartbeat_front_addr":"172.21.15.80:6821/976514285","state":["exists","up"]},{"osd":5,"uuid":"ed10c84f-8c39-41cc-9b64-7c482a94a48e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6825","nonce":627153447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6827","nonce":627153447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6831","nonce":627153447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6829","nonce":627153447}]},"public_addr":"172.21.15.80:6825/627153447","cluster_addr":"172.21.15.80:6827/627153447","heartbeat_back_addr":"172.21.15.80:6831/627153447","heartbeat_front_addr":"172.21.15.80:6829/627153447","state":["exists","up"]},{"osd":6,"uuid":"c7a3538a-7048-4fba-8781-b440f03c25db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6803","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6805","nonce":488342527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6807","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6809","nonce":488342527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6815","nonce":488342527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6811","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6813","nonce":488342527}]},"public_addr":"172.21.15.80:6805/488342527","cluster_addr":"172.21.15.80:6809/488342527","heartbeat_back_addr":"172.21.15.80:6815/488342527","heartbeat_front_addr":"172.21.15.80:6813/488342527","state":["exists","up"]},{"osd":7,"uuid":"f9418104-3a97-4110-bc3a-1f8151d0f757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6801","nonce":982831666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6804","nonce":982831666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6812","nonce":982831666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6808","nonce":982831666}]},"public_addr":"172.21.15.80:6801/982831666","cluster_addr":"172.21.15.80:6804/982831666","heartbeat_back_addr":"172.21.15.80:6812/982831666","heartbeat_front_addr":"172.21.15.80:6808/982831666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.970828+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.858116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.987556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.864603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.756293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.591502+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:33:58.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.129+0000 7fadbe076640 1 -- 172.21.15.5:0/3051589515 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fada4067660 msgr2=0x7fada4069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:58.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.129+0000 7fadbe076640 1 --2- 172.21.15.5:0/3051589515 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fada4067660 0x7fada4069b20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fadac007fd0 tx=0x7fadac007ba0 comp rx=0 tx=0).stop 2024-03-18T15:33:58.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.129+0000 7fadbe076640 1 -- 172.21.15.5:0/3051589515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadb813f260 msgr2=0x7fadb81408f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:58.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.129+0000 7fadbe076640 1 --2- 172.21.15.5:0/3051589515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadb813f260 0x7fadb81408f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fada800c500 tx=0x7fada800c9c0 comp rx=0 tx=0).stop 2024-03-18T15:33:58.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.129+0000 7fadbe076640 1 -- 172.21.15.5:0/3051589515 shutdown_connections 2024-03-18T15:33:58.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.129+0000 7fadbe076640 1 --2- 172.21.15.5:0/3051589515 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fada4067660 0x7fada4069b20 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:58.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.129+0000 7fadbe076640 1 --2- 172.21.15.5:0/3051589515 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fadb813f6a0 0x7fadb8140e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:58.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.129+0000 7fadbe076640 1 --2- 172.21.15.5:0/3051589515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fadb813f260 0x7fadb81408f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:58.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.129+0000 7fadbe076640 1 --2- 172.21.15.5:0/3051589515 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fadb806d2d0 0x7fadb8145ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:58.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.129+0000 7fadbe076640 1 -- 172.21.15.5:0/3051589515 >> 172.21.15.5:0/3051589515 conn(0x7fadb80b5670 msgr2=0x7fadb813cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:58.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.129+0000 7fadbe076640 1 -- 172.21.15.5:0/3051589515 shutdown_connections 2024-03-18T15:33:58.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.129+0000 7fadbe076640 1 -- 172.21.15.5:0/3051589515 wait complete. 2024-03-18T15:33:58.151 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T15:33:28.963186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-18T15:33:40.533646+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-18T15:33:41.323519+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T15:33:58.152 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T15:33:58.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.281+0000 7f4358c1f640 1 Processor -- start 2024-03-18T15:33:58.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.281+0000 7f4358c1f640 1 -- start start 2024-03-18T15:33:58.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.281+0000 7f4358c1f640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4354070d00 0x7f4354071160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:58.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.281+0000 7f4358c1f640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f435413dfa0 0x7f435413e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:58.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.281+0000 7f4358c1f640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f435413e970 con 0x7f43541479a0 2024-03-18T15:33:58.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.281+0000 7f4358c1f640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f43540702b0 con 0x7f4354070d00 2024-03-18T15:33:58.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.281+0000 7f4358c1f640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f4354070870 con 0x7f435413dfa0 2024-03-18T15:33:58.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.281+0000 7f4351d9b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4354070d00 0x7f4354071160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:58.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.281+0000 7f4351d9b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f4354070d00 0x7f4354071160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43126/0 (socket says 172.21.15.5:43126) 2024-03-18T15:33:58.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.281+0000 7f4351d9b640 1 -- 172.21.15.5:0/1453468664 learned_addr learned my addr 172.21.15.5:0/1453468664 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:58.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.281+0000 7f4352d9d640 1 --2- 172.21.15.5:0/1453468664 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f435413dfa0 0x7f435413e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:58.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.281+0000 7f4351d9b640 1 -- 172.21.15.5:0/1453468664 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f435413dfa0 msgr2=0x7f435413e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:58.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.281+0000 7f4351d9b640 1 --2- 172.21.15.5:0/1453468664 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f435413dfa0 0x7f435413e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:58.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.281+0000 7f4351d9b640 1 -- 172.21.15.5:0/1453468664 >> v1:172.21.15.80:6789/0 conn(0x7f43541479a0 legacy=0x7f4354147d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:58.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.281+0000 7f4351d9b640 1 -- 172.21.15.5:0/1453468664 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43540716a0 con 0x7f4354070d00 2024-03-18T15:33:58.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.285+0000 7f4351d9b640 1 --2- 172.21.15.5:0/1453468664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4354070d00 0x7f4354071160 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f433c009870 tx=0x7f433c01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0707dc830e509b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:58.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.285+0000 7f435159a640 1 -- 172.21.15.5:0/1453468664 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433c0b0070 con 0x7f4354070d00 2024-03-18T15:33:58.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.285+0000 7f435159a640 1 -- 172.21.15.5:0/1453468664 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f433c0ab600 con 0x7f4354070d00 2024-03-18T15:33:58.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.285+0000 7f435159a640 1 -- 172.21.15.5:0/1453468664 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433c0ab9d0 con 0x7f4354070d00 2024-03-18T15:33:58.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.285+0000 7f4358c1f640 1 -- 172.21.15.5:0/1453468664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4354070d00 msgr2=0x7f4354071160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:58.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.285+0000 7f4358c1f640 1 --2- 172.21.15.5:0/1453468664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4354070d00 0x7f4354071160 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f433c009870 tx=0x7f433c01c440 comp rx=0 tx=0).stop 2024-03-18T15:33:58.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.285+0000 7f4358c1f640 1 -- 172.21.15.5:0/1453468664 shutdown_connections 2024-03-18T15:33:58.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.285+0000 7f4358c1f640 1 --2- 172.21.15.5:0/1453468664 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f435413dfa0 0x7f435413e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:58.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.285+0000 7f4358c1f640 1 --2- 172.21.15.5:0/1453468664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4354070d00 0x7f4354071160 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:58.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.285+0000 7f4358c1f640 1 -- 172.21.15.5:0/1453468664 >> 172.21.15.5:0/1453468664 conn(0x7f43540b7a80 msgr2=0x7f43540b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:58.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.285+0000 7f4358c1f640 1 -- 172.21.15.5:0/1453468664 shutdown_connections 2024-03-18T15:33:58.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.285+0000 7f4358c1f640 1 -- 172.21.15.5:0/1453468664 wait complete. 2024-03-18T15:33:58.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.285+0000 7f4358c1f640 1 Processor -- start 2024-03-18T15:33:58.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.285+0000 7f4358c1f640 1 -- start start 2024-03-18T15:33:58.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.285+0000 7f4358c1f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4354070d00 0x7f435413bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:58.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.285+0000 7f4358c1f640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f435413dfa0 0x7f43541e0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:58.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.285+0000 7f4358c1f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43541479a0 0x7f4354138240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:58.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.285+0000 7f4358c1f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4354151410 con 0x7f4354070d00 2024-03-18T15:33:58.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.285+0000 7f4358c1f640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4354151290 con 0x7f43541479a0 2024-03-18T15:33:58.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.285+0000 7f4358c1f640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f4354151590 con 0x7f435413dfa0 2024-03-18T15:33:58.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.285+0000 7f435259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4354070d00 0x7f435413bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:58.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.285+0000 7f435259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4354070d00 0x7f435413bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43138/0 (socket says 172.21.15.5:43138) 2024-03-18T15:33:58.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.285+0000 7f435259c640 1 -- 172.21.15.5:0/2162979304 learned_addr learned my addr 172.21.15.5:0/2162979304 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:58.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.285+0000 7f4351d9b640 1 --2- 172.21.15.5:0/2162979304 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f435413dfa0 0x7f43541e0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:58.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.289+0000 7f4352d9d640 1 --2- 172.21.15.5:0/2162979304 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43541479a0 0x7f4354138240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:58.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.289+0000 7f435259c640 1 -- 172.21.15.5:0/2162979304 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f435413dfa0 msgr2=0x7f43541e0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:58.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.289+0000 7f435259c640 1 --2- 172.21.15.5:0/2162979304 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f435413dfa0 0x7f43541e0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:58.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.289+0000 7f435259c640 1 -- 172.21.15.5:0/2162979304 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43541479a0 msgr2=0x7f4354138240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:58.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.289+0000 7f435259c640 1 --2- 172.21.15.5:0/2162979304 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43541479a0 0x7f4354138240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:58.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.289+0000 7f435259c640 1 -- 172.21.15.5:0/2162979304 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4354146150 con 0x7f4354070d00 2024-03-18T15:33:58.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.289+0000 7f435259c640 1 --2- 172.21.15.5:0/2162979304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4354070d00 0x7f435413bd40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f434c00c370 tx=0x7f434c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:58.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.289+0000 7f434affd640 1 -- 172.21.15.5:0/2162979304 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c0052d0 con 0x7f4354070d00 2024-03-18T15:33:58.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.289+0000 7f434affd640 1 -- 172.21.15.5:0/2162979304 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f434c005470 con 0x7f4354070d00 2024-03-18T15:33:58.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.289+0000 7f4358c1f640 1 -- 172.21.15.5:0/2162979304 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4354146d80 con 0x7f4354070d00 2024-03-18T15:33:58.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.289+0000 7f434affd640 1 -- 172.21.15.5:0/2162979304 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c007260 con 0x7f4354070d00 2024-03-18T15:33:58.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.289+0000 7f4358c1f640 1 -- 172.21.15.5:0/2162979304 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43541e1250 con 0x7f4354070d00 2024-03-18T15:33:58.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.289+0000 7f4358c1f640 1 -- 172.21.15.5:0/2162979304 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f435404f550 con 0x7f4354070d00 2024-03-18T15:33:58.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.289+0000 7f434affd640 1 -- 172.21.15.5:0/2162979304 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f434c007400 con 0x7f4354070d00 2024-03-18T15:33:58.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.289+0000 7f434affd640 1 --2- 172.21.15.5:0/2162979304 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f4334067660 0x7f4334069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:58.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.289+0000 7f434affd640 1 -- 172.21.15.5:0/2162979304 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f434c084c80 con 0x7f4354070d00 2024-03-18T15:33:58.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.289+0000 7f4351d9b640 1 --2- 172.21.15.5:0/2162979304 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f4334067660 0x7f4334069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:58.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.289+0000 7f4351d9b640 1 --2- 172.21.15.5:0/2162979304 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f4334067660 0x7f4334069b20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f433c0ad000 tx=0x7f433c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:58.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.293+0000 7f434affd640 1 -- 172.21.15.5:0/2162979304 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f434c055990 con 0x7f4354070d00 2024-03-18T15:33:58.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.457+0000 7f4358c1f640 1 -- 172.21.15.5:0/2162979304 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f43540b4d90 con 0x7f4354070d00 2024-03-18T15:33:58.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.457+0000 7f434affd640 1 -- 172.21.15.5:0/2162979304 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f434c055290 con 0x7f4354070d00 2024-03-18T15:33:58.458 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-03-18T15:33:58.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.457+0000 7f4358c1f640 1 -- 172.21.15.5:0/2162979304 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f4334067660 msgr2=0x7f4334069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:58.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.457+0000 7f4358c1f640 1 --2- 172.21.15.5:0/2162979304 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f4334067660 0x7f4334069b20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f433c0ad000 tx=0x7f433c008040 comp rx=0 tx=0).stop 2024-03-18T15:33:58.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.457+0000 7f4358c1f640 1 -- 172.21.15.5:0/2162979304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4354070d00 msgr2=0x7f435413bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:58.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.457+0000 7f4358c1f640 1 --2- 172.21.15.5:0/2162979304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4354070d00 0x7f435413bd40 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f434c00c370 tx=0x7f434c00c830 comp rx=0 tx=0).stop 2024-03-18T15:33:58.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.457+0000 7f4358c1f640 1 -- 172.21.15.5:0/2162979304 shutdown_connections 2024-03-18T15:33:58.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.457+0000 7f4358c1f640 1 --2- 172.21.15.5:0/2162979304 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f4334067660 0x7f4334069b20 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:58.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.457+0000 7f4358c1f640 1 --2- 172.21.15.5:0/2162979304 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43541479a0 0x7f4354138240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:58.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.457+0000 7f4358c1f640 1 --2- 172.21.15.5:0/2162979304 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f435413dfa0 0x7f43541e0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:58.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.457+0000 7f4358c1f640 1 --2- 172.21.15.5:0/2162979304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4354070d00 0x7f435413bd40 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:58.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.457+0000 7f4358c1f640 1 -- 172.21.15.5:0/2162979304 >> 172.21.15.5:0/2162979304 conn(0x7f43540b7a80 msgr2=0x7f435406d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:58.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.457+0000 7f4358c1f640 1 -- 172.21.15.5:0/2162979304 shutdown_connections 2024-03-18T15:33:58.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.457+0000 7f4358c1f640 1 -- 172.21.15.5:0/2162979304 wait complete. 2024-03-18T15:33:58.477 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-18T15:33:58.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.597+0000 7f726e606640 1 Processor -- start 2024-03-18T15:33:58.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.597+0000 7f726e606640 1 -- start start 2024-03-18T15:33:58.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.597+0000 7f726e606640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f726813f260 0x7f726813f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:58.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.597+0000 7f726e606640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72680b2ab0 con 0x7f726806d2d0 2024-03-18T15:33:58.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.597+0000 7f726e606640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72680b0ea0 con 0x7f726814dc70 2024-03-18T15:33:58.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.597+0000 7f726e606640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f72680728c0 con 0x7f726813f260 2024-03-18T15:33:58.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.597+0000 7f7267fff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f726806d2d0 0x7f72680b23a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46946/0 (socket says 172.21.15.5:46946) 2024-03-18T15:33:58.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.597+0000 7f7267fff640 1 -- 172.21.15.5:0/9677003 learned_addr learned my addr 172.21.15.5:0/9677003 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:33:58.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.597+0000 7f72677fe640 1 --2- 172.21.15.5:0/9677003 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f726813f260 0x7f726813f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:58.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.597+0000 7f7266ffd640 1 -- 172.21.15.5:0/9677003 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2531443697 0 0) 0x7f72680b2ab0 con 0x7f726806d2d0 2024-03-18T15:33:58.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.597+0000 7f7266ffd640 1 -- 172.21.15.5:0/9677003 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7250003530 con 0x7f726806d2d0 2024-03-18T15:33:58.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.597+0000 7f7266ffd640 1 -- 172.21.15.5:0/9677003 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3861629717 0 0) 0x7f7250003530 con 0x7f726806d2d0 2024-03-18T15:33:58.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.597+0000 7f7266ffd640 1 -- 172.21.15.5:0/9677003 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f726813f260 msgr2=0x7f726813f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:58.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.597+0000 7f7266ffd640 1 --2- 172.21.15.5:0/9677003 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f726813f260 0x7f726813f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:58.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.597+0000 7f7266ffd640 1 -- 172.21.15.5:0/9677003 >> v1:172.21.15.80:6789/0 conn(0x7f726814dc70 legacy=0x7f7268150060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:58.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.597+0000 7f7266ffd640 1 -- 172.21.15.5:0/9677003 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f726806d6b0 con 0x7f726806d2d0 2024-03-18T15:33:58.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.597+0000 7f7266ffd640 1 -- 172.21.15.5:0/9677003 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3837549003 0 0) 0x7f7258001680 con 0x7f726806d2d0 2024-03-18T15:33:58.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.597+0000 7f7266ffd640 1 -- 172.21.15.5:0/9677003 >> v1:172.21.15.5:6789/0 conn(0x7f726806d2d0 legacy=0x7f72680b23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:58.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.597+0000 7f7266ffd640 1 --2- 172.21.15.5:0/9677003 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7250003ad0 0x7f7250003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:58.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.597+0000 7f7266ffd640 1 --2- 172.21.15.5:0/9677003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7250004810 0x7f7250004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:58.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.597+0000 7f7266ffd640 1 --2- 172.21.15.5:0/9677003 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f726813f260 0x7f725000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:58.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.597+0000 7f7266ffd640 1 -- 172.21.15.5:0/9677003 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f72680728c0 con 0x7f7250004810 2024-03-18T15:33:58.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.597+0000 7f7266ffd640 1 -- 172.21.15.5:0/9677003 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7250005530 con 0x7f7250003ad0 2024-03-18T15:33:58.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.597+0000 7f7266ffd640 1 -- 172.21.15.5:0/9677003 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f725000bf10 con 0x7f726813f260 2024-03-18T15:33:58.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.597+0000 7f72677fe640 1 --2- 172.21.15.5:0/9677003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7250004810 0x7f7250004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:58.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.597+0000 7f7267fff640 1 --2- 172.21.15.5:0/9677003 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7250003ad0 0x7f7250003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:58.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.597+0000 7f726cba3640 1 --2- 172.21.15.5:0/9677003 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f726813f260 0x7f725000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:58.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.597+0000 7f72677fe640 1 -- 172.21.15.5:0/9677003 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f726813f260 msgr2=0x7f725000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:58.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.597+0000 7f72677fe640 1 --2- 172.21.15.5:0/9677003 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f726813f260 0x7f725000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:58.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.597+0000 7f72677fe640 1 -- 172.21.15.5:0/9677003 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7250003ad0 msgr2=0x7f7250003ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:58.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.597+0000 7f72677fe640 1 --2- 172.21.15.5:0/9677003 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7250003ad0 0x7f7250003ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:58.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.597+0000 7f72677fe640 1 -- 172.21.15.5:0/9677003 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7250003530 con 0x7f7250004810 2024-03-18T15:33:58.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.601+0000 7f72677fe640 1 --2- 172.21.15.5:0/9677003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7250004810 0x7f7250004c90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f725c009ae0 tx=0x7f725c01ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a0117f1f39c4fd server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:58.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.601+0000 7f7266ffd640 1 -- 172.21.15.5:0/9677003 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725c0b2070 con 0x7f7250004810 2024-03-18T15:33:58.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.601+0000 7f7266ffd640 1 -- 172.21.15.5:0/9677003 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f725c0a5610 con 0x7f7250004810 2024-03-18T15:33:58.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.601+0000 7f7266ffd640 1 -- 172.21.15.5:0/9677003 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725c0096b0 con 0x7f7250004810 2024-03-18T15:33:58.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.601+0000 7f726e606640 1 -- 172.21.15.5:0/9677003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7250004810 msgr2=0x7f7250004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:58.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.601+0000 7f726e606640 1 --2- 172.21.15.5:0/9677003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7250004810 0x7f7250004c90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f725c009ae0 tx=0x7f725c01ca40 comp rx=0 tx=0).stop 2024-03-18T15:33:58.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.601+0000 7f726cba3640 1 -- 172.21.15.5:0/9677003 reap_dead start 2024-03-18T15:33:58.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.601+0000 7f726e606640 1 -- 172.21.15.5:0/9677003 shutdown_connections 2024-03-18T15:33:58.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.601+0000 7f726e606640 1 -- 172.21.15.5:0/9677003 >> 172.21.15.5:0/9677003 conn(0x7f72680b5670 msgr2=0x7f72680b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:58.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.601+0000 7f726e606640 1 -- 172.21.15.5:0/9677003 shutdown_connections 2024-03-18T15:33:58.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.601+0000 7f726e606640 1 -- 172.21.15.5:0/9677003 wait complete. 2024-03-18T15:33:58.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.601+0000 7f726e606640 1 Processor -- start 2024-03-18T15:33:58.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.601+0000 7f726e606640 1 -- start start 2024-03-18T15:33:58.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.601+0000 7f726e606640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7250004810 0x7f7268145610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:58.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.601+0000 7f726e606640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7268144040 0x7f7268071350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:58.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.601+0000 7f726e606640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f72681444c0 0x7f7268071890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:58.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.601+0000 7f726e606640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f72680b01f0 con 0x7f7250004810 2024-03-18T15:33:58.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.601+0000 7f726e606640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f72680b0070 con 0x7f72681444c0 2024-03-18T15:33:58.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.601+0000 7f726e606640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f72680b07e0 con 0x7f7268144040 2024-03-18T15:33:58.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.601+0000 7f7267fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7250004810 0x7f7268145610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:58.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.601+0000 7f7267fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7250004810 0x7f7268145610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43148/0 (socket says 172.21.15.5:43148) 2024-03-18T15:33:58.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.601+0000 7f7267fff640 1 -- 172.21.15.5:0/3035777557 learned_addr learned my addr 172.21.15.5:0/3035777557 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:58.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.601+0000 7f72677fe640 1 --2- 172.21.15.5:0/3035777557 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7268144040 0x7f7268071350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:58.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.601+0000 7f726cba3640 1 --2- 172.21.15.5:0/3035777557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f72681444c0 0x7f7268071890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:58.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.601+0000 7f7267fff640 1 -- 172.21.15.5:0/3035777557 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7268144040 msgr2=0x7f7268071350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:58.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.601+0000 7f7267fff640 1 --2- 172.21.15.5:0/3035777557 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7268144040 0x7f7268071350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:58.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.601+0000 7f7267fff640 1 -- 172.21.15.5:0/3035777557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f72681444c0 msgr2=0x7f7268071890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:58.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.601+0000 7f7267fff640 1 --2- 172.21.15.5:0/3035777557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f72681444c0 0x7f7268071890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:58.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.601+0000 7f7267fff640 1 -- 172.21.15.5:0/3035777557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f726806f8d0 con 0x7f7250004810 2024-03-18T15:33:58.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.601+0000 7f726cba3640 1 --2- 172.21.15.5:0/3035777557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f72681444c0 0x7f7268071890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:33:58.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.605+0000 7f7267fff640 1 --2- 172.21.15.5:0/3035777557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7250004810 0x7f7268145610 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f725800e700 tx=0x7f725800ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:58.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.605+0000 7f7264ff9640 1 -- 172.21.15.5:0/3035777557 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725800c6c0 con 0x7f7250004810 2024-03-18T15:33:58.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.605+0000 7f7264ff9640 1 -- 172.21.15.5:0/3035777557 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7258018070 con 0x7f7250004810 2024-03-18T15:33:58.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.605+0000 7f726e606640 1 -- 172.21.15.5:0/3035777557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f726806fb40 con 0x7f7250004810 2024-03-18T15:33:58.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.605+0000 7f7264ff9640 1 -- 172.21.15.5:0/3035777557 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725800cac0 con 0x7f7250004810 2024-03-18T15:33:58.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.605+0000 7f726e606640 1 -- 172.21.15.5:0/3035777557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72681e04c0 con 0x7f7250004810 2024-03-18T15:33:58.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.605+0000 7f726e606640 1 -- 172.21.15.5:0/3035777557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f726804f550 con 0x7f7250004810 2024-03-18T15:33:58.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.605+0000 7f7264ff9640 1 -- 172.21.15.5:0/3035777557 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f725800cc60 con 0x7f7250004810 2024-03-18T15:33:58.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.605+0000 7f7264ff9640 1 --2- 172.21.15.5:0/3035777557 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f7254067660 0x7f7254069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:58.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.605+0000 7f7264ff9640 1 -- 172.21.15.5:0/3035777557 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7258085700 con 0x7f7250004810 2024-03-18T15:33:58.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.605+0000 7f72677fe640 1 --2- 172.21.15.5:0/3035777557 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f7254067660 0x7f7254069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:58.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.605+0000 7f72677fe640 1 --2- 172.21.15.5:0/3035777557 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f7254067660 0x7f7254069b20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f725c0af000 tx=0x7f725c0046d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:58.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.609+0000 7f7264ff9640 1 -- 172.21.15.5:0/3035777557 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f7258056370 con 0x7f7250004810 2024-03-18T15:33:58.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.761+0000 7f726e606640 1 -- 172.21.15.5:0/3035777557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f726813c400 con 0x7f7250004810 2024-03-18T15:33:58.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.765+0000 7f7264ff9640 1 -- 172.21.15.5:0/3035777557 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f7258055c70 con 0x7f7250004810 2024-03-18T15:33:58.766 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-03-18T15:33:58.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.765+0000 7f726e606640 1 -- 172.21.15.5:0/3035777557 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f7254067660 msgr2=0x7f7254069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:58.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.765+0000 7f726e606640 1 --2- 172.21.15.5:0/3035777557 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f7254067660 0x7f7254069b20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f725c0af000 tx=0x7f725c0046d0 comp rx=0 tx=0).stop 2024-03-18T15:33:58.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.765+0000 7f726e606640 1 -- 172.21.15.5:0/3035777557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7250004810 msgr2=0x7f7268145610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:58.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.765+0000 7f726e606640 1 --2- 172.21.15.5:0/3035777557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7250004810 0x7f7268145610 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f725800e700 tx=0x7f725800ebc0 comp rx=0 tx=0).stop 2024-03-18T15:33:58.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.765+0000 7f726e606640 1 -- 172.21.15.5:0/3035777557 shutdown_connections 2024-03-18T15:33:58.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.765+0000 7f726e606640 1 --2- 172.21.15.5:0/3035777557 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f7254067660 0x7f7254069b20 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:58.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.765+0000 7f726e606640 1 --2- 172.21.15.5:0/3035777557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f72681444c0 0x7f7268071890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:58.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.765+0000 7f726e606640 1 --2- 172.21.15.5:0/3035777557 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7268144040 0x7f7268071350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:58.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.765+0000 7f726e606640 1 --2- 172.21.15.5:0/3035777557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7250004810 0x7f7268145610 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:58.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.765+0000 7f726e606640 1 -- 172.21.15.5:0/3035777557 >> 172.21.15.5:0/3035777557 conn(0x7f72680b5670 msgr2=0x7f726813d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:58.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.765+0000 7f726e606640 1 -- 172.21.15.5:0/3035777557 shutdown_connections 2024-03-18T15:33:58.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.765+0000 7f726e606640 1 -- 172.21.15.5:0/3035777557 wait complete. 2024-03-18T15:33:58.783 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-18T15:33:58.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.909+0000 7f0dca054640 1 Processor -- start 2024-03-18T15:33:58.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.909+0000 7f0dca054640 1 -- start start 2024-03-18T15:33:58.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.909+0000 7f0dca054640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0dc413e450 0x7f0dc40afad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:58.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.909+0000 7f0dca054640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0dc40af4f0 0x7f0dc414ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:58.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.909+0000 7f0dca054640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0dc40b9c20 con 0x7f0dc40b0010 2024-03-18T15:33:58.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.909+0000 7f0dca054640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f0dc40b99d0 con 0x7f0dc40af4f0 2024-03-18T15:33:58.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.909+0000 7f0dca054640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f0dc40b9f00 con 0x7f0dc413e450 2024-03-18T15:33:58.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.909+0000 7f0dc3fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0dc40af4f0 0x7f0dc414ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:58.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.909+0000 7f0dc2ffd640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f0dc40b0010 0x7f0dc41442e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46956/0 (socket says 172.21.15.5:46956) 2024-03-18T15:33:58.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.909+0000 7f0dc2ffd640 1 -- 172.21.15.5:0/2871060127 learned_addr learned my addr 172.21.15.5:0/2871060127 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:33:58.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.909+0000 7f0dc3fff640 1 --2- 172.21.15.5:0/2871060127 >> v2:172.21.15.5:3300/0 conn(0x7f0dc40af4f0 0x7f0dc414ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43160/0 (socket says 172.21.15.5:43160) 2024-03-18T15:33:58.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.909+0000 7f0dc37fe640 1 --2- 172.21.15.5:0/2871060127 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0dc413e450 0x7f0dc40afad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:58.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.909+0000 7f0dc3fff640 1 -- 172.21.15.5:0/2871060127 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0dc413e450 msgr2=0x7f0dc40afad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:58.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.909+0000 7f0dc3fff640 1 --2- 172.21.15.5:0/2871060127 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0dc413e450 0x7f0dc40afad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:58.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.909+0000 7f0dc3fff640 1 -- 172.21.15.5:0/2871060127 >> v1:172.21.15.5:6789/0 conn(0x7f0dc40b0010 legacy=0x7f0dc41442e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:58.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.909+0000 7f0dc3fff640 1 -- 172.21.15.5:0/2871060127 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dc413e830 con 0x7f0dc40af4f0 2024-03-18T15:33:58.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.913+0000 7f0dc3fff640 1 --2- 172.21.15.5:0/2871060127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dc40af4f0 0x7f0dc414ea10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc009d90 tx=0x7f0dbc01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16fdb68bf9c0c1b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:58.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.913+0000 7f0dc27fc640 1 -- 172.21.15.5:0/2871060127 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc0b2070 con 0x7f0dc40af4f0 2024-03-18T15:33:58.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.913+0000 7f0dc27fc640 1 -- 172.21.15.5:0/2871060127 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dbc0a59c0 con 0x7f0dc40af4f0 2024-03-18T15:33:58.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.913+0000 7f0dc27fc640 1 -- 172.21.15.5:0/2871060127 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc0a5d90 con 0x7f0dc40af4f0 2024-03-18T15:33:58.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.913+0000 7f0dca054640 1 -- 172.21.15.5:0/2871060127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dc40af4f0 msgr2=0x7f0dc414ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:58.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.913+0000 7f0dca054640 1 --2- 172.21.15.5:0/2871060127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dc40af4f0 0x7f0dc414ea10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc009d90 tx=0x7f0dbc01c8e0 comp rx=0 tx=0).stop 2024-03-18T15:33:58.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.913+0000 7f0dca054640 1 -- 172.21.15.5:0/2871060127 shutdown_connections 2024-03-18T15:33:58.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.913+0000 7f0dca054640 1 --2- 172.21.15.5:0/2871060127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dc40af4f0 0x7f0dc414ea10 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:58.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.913+0000 7f0dca054640 1 --2- 172.21.15.5:0/2871060127 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0dc413e450 0x7f0dc40afad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:58.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.913+0000 7f0dca054640 1 -- 172.21.15.5:0/2871060127 >> 172.21.15.5:0/2871060127 conn(0x7f0dc406e230 msgr2=0x7f0dc406e640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:58.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.913+0000 7f0dca054640 1 -- 172.21.15.5:0/2871060127 shutdown_connections 2024-03-18T15:33:58.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.913+0000 7f0dca054640 1 -- 172.21.15.5:0/2871060127 wait complete. 2024-03-18T15:33:58.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.913+0000 7f0dca054640 1 Processor -- start 2024-03-18T15:33:58.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.913+0000 7f0dca054640 1 -- start start 2024-03-18T15:33:58.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.913+0000 7f0dca054640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0dc40af4f0 0x7f0dc41e3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:58.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.913+0000 7f0dca054640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0dc413e450 0x7f0dc4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:58.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.913+0000 7f0dca054640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dc40b3ca0 0x7f0dc409f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:58.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.913+0000 7f0dca054640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc4151b00 con 0x7f0dc40b3ca0 2024-03-18T15:33:58.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.913+0000 7f0dca054640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc4151980 con 0x7f0dc40af4f0 2024-03-18T15:33:58.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.913+0000 7f0dca054640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f0dc4151c80 con 0x7f0dc413e450 2024-03-18T15:33:58.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.913+0000 7f0dc3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dc40b3ca0 0x7f0dc409f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:58.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.913+0000 7f0dc3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dc40b3ca0 0x7f0dc409f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43166/0 (socket says 172.21.15.5:43166) 2024-03-18T15:33:58.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.913+0000 7f0dc3fff640 1 -- 172.21.15.5:0/2100989211 learned_addr learned my addr 172.21.15.5:0/2100989211 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:58.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.913+0000 7f0dc2ffd640 1 --2- 172.21.15.5:0/2100989211 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0dc413e450 0x7f0dc4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:58.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.917+0000 7f0dc37fe640 1 --2- 172.21.15.5:0/2100989211 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0dc40af4f0 0x7f0dc41e3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:58.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.917+0000 7f0dc3fff640 1 -- 172.21.15.5:0/2100989211 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0dc413e450 msgr2=0x7f0dc4071940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:58.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.917+0000 7f0dc3fff640 1 --2- 172.21.15.5:0/2100989211 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0dc413e450 0x7f0dc4071940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:58.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.917+0000 7f0dc3fff640 1 -- 172.21.15.5:0/2100989211 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0dc40af4f0 msgr2=0x7f0dc41e3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:58.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.917+0000 7f0dc3fff640 1 --2- 172.21.15.5:0/2100989211 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0dc40af4f0 0x7f0dc41e3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:58.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.917+0000 7f0dc3fff640 1 -- 172.21.15.5:0/2100989211 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dc413f280 con 0x7f0dc40b3ca0 2024-03-18T15:33:58.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.917+0000 7f0dc3fff640 1 --2- 172.21.15.5:0/2100989211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dc40b3ca0 0x7f0dc409f0d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc009d90 tx=0x7f0dbc019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:58.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.917+0000 7f0dbbfff640 1 -- 172.21.15.5:0/2100989211 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc0b2050 con 0x7f0dc40b3ca0 2024-03-18T15:33:58.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.917+0000 7f0dbbfff640 1 -- 172.21.15.5:0/2100989211 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dbc0be070 con 0x7f0dc40b3ca0 2024-03-18T15:33:58.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.917+0000 7f0dbbfff640 1 -- 172.21.15.5:0/2100989211 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dbc0adb10 con 0x7f0dc40b3ca0 2024-03-18T15:33:58.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.917+0000 7f0dca054640 1 -- 172.21.15.5:0/2100989211 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dc4072090 con 0x7f0dc40b3ca0 2024-03-18T15:33:58.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.917+0000 7f0dca054640 1 -- 172.21.15.5:0/2100989211 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dc40710c0 con 0x7f0dc40b3ca0 2024-03-18T15:33:58.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.917+0000 7f0dca054640 1 -- 172.21.15.5:0/2100989211 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dc404f550 con 0x7f0dc40b3ca0 2024-03-18T15:33:58.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.917+0000 7f0dbbfff640 1 -- 172.21.15.5:0/2100989211 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f0dbc0adcb0 con 0x7f0dc40b3ca0 2024-03-18T15:33:58.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.917+0000 7f0dbbfff640 1 --2- 172.21.15.5:0/2100989211 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0dac067660 0x7f0dac069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:58.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.917+0000 7f0dc37fe640 1 --2- 172.21.15.5:0/2100989211 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0dac067660 0x7f0dac069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:58.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.917+0000 7f0dbbfff640 1 -- 172.21.15.5:0/2100989211 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0dbc120bb0 con 0x7f0dc40b3ca0 2024-03-18T15:33:58.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.917+0000 7f0dc37fe640 1 --2- 172.21.15.5:0/2100989211 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0dac067660 0x7f0dac069b20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0dc413eed0 tx=0x7f0db4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:58.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:58.921+0000 7f0dbbfff640 1 -- 172.21.15.5:0/2100989211 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f0dbc016560 con 0x7f0dc40b3ca0 2024-03-18T15:33:59.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.089+0000 7f0dca054640 1 -- 172.21.15.5:0/2100989211 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0dc4142460 con 0x7f0dc40b3ca0 2024-03-18T15:33:59.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.089+0000 7f0dbbfff640 1 -- 172.21.15.5:0/2100989211 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f0dbc0ab340 con 0x7f0dc40b3ca0 2024-03-18T15:33:59.090 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-03-18T15:33:59.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.089+0000 7f0dca054640 1 -- 172.21.15.5:0/2100989211 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0dac067660 msgr2=0x7f0dac069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:59.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.089+0000 7f0dca054640 1 --2- 172.21.15.5:0/2100989211 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0dac067660 0x7f0dac069b20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0dc413eed0 tx=0x7f0db4008040 comp rx=0 tx=0).stop 2024-03-18T15:33:59.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.089+0000 7f0dca054640 1 -- 172.21.15.5:0/2100989211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dc40b3ca0 msgr2=0x7f0dc409f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:59.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.089+0000 7f0dca054640 1 --2- 172.21.15.5:0/2100989211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dc40b3ca0 0x7f0dc409f0d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc009d90 tx=0x7f0dbc019680 comp rx=0 tx=0).stop 2024-03-18T15:33:59.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.089+0000 7f0dca054640 1 -- 172.21.15.5:0/2100989211 shutdown_connections 2024-03-18T15:33:59.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.089+0000 7f0dca054640 1 --2- 172.21.15.5:0/2100989211 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0dac067660 0x7f0dac069b20 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:59.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.089+0000 7f0dca054640 1 --2- 172.21.15.5:0/2100989211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0dc40b3ca0 0x7f0dc409f0d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:59.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.089+0000 7f0dca054640 1 --2- 172.21.15.5:0/2100989211 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0dc413e450 0x7f0dc4071940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:59.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.089+0000 7f0dca054640 1 --2- 172.21.15.5:0/2100989211 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0dc40af4f0 0x7f0dc41e3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:59.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.089+0000 7f0dca054640 1 -- 172.21.15.5:0/2100989211 >> 172.21.15.5:0/2100989211 conn(0x7f0dc406e230 msgr2=0x7f0dc4142980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:59.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.089+0000 7f0dca054640 1 -- 172.21.15.5:0/2100989211 shutdown_connections 2024-03-18T15:33:59.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.089+0000 7f0dca054640 1 -- 172.21.15.5:0/2100989211 wait complete. 2024-03-18T15:33:59.110 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:33:59.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.225+0000 7f9e9f42c640 1 Processor -- start 2024-03-18T15:33:59.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.229+0000 7f9e9f42c640 1 -- start start 2024-03-18T15:33:59.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.229+0000 7f9e9f42c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f9e980b6d40 0x7f9e98148a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:59.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.229+0000 7f9e9f42c640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9e98148f90 0x7f9e980b4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:59.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.229+0000 7f9e9f42c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9e980b4c70 0x7f9e98071380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:59.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.229+0000 7f9e9f42c640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f9e98149440 con 0x7f9e980b4c70 2024-03-18T15:33:59.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.229+0000 7f9e9f42c640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f9e980b78e0 con 0x7f9e980b6d40 2024-03-18T15:33:59.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.229+0000 7f9e9f42c640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f9e980b7120 con 0x7f9e98148f90 2024-03-18T15:33:59.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.229+0000 7f9e9d9c9640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9e980b4c70 0x7f9e98071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:59.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.229+0000 7f9e9d9c9640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9e980b4c70 0x7f9e98071380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43172/0 (socket says 172.21.15.5:43172) 2024-03-18T15:33:59.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.229+0000 7f9e9d9c9640 1 -- 172.21.15.5:0/1871938785 learned_addr learned my addr 172.21.15.5:0/1871938785 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:59.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.229+0000 7f9e9c9c7640 1 --2- 172.21.15.5:0/1871938785 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9e98148f90 0x7f9e980b4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:59.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.229+0000 7f9e9d1c8640 1 --2- 172.21.15.5:0/1871938785 >> v2:172.21.15.80:3300/0 conn(0x7f9e980b6d40 0x7f9e98148a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:59.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.229+0000 7f9e9d9c9640 1 -- 172.21.15.5:0/1871938785 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9e98148f90 msgr2=0x7f9e980b4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:59.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.229+0000 7f9e9d9c9640 1 --2- 172.21.15.5:0/1871938785 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9e98148f90 0x7f9e980b4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:59.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.229+0000 7f9e9d9c9640 1 -- 172.21.15.5:0/1871938785 >> v2:172.21.15.80:3300/0 conn(0x7f9e980b6d40 msgr2=0x7f9e98148a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:59.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.229+0000 7f9e9d9c9640 1 --2- 172.21.15.5:0/1871938785 >> v2:172.21.15.80:3300/0 conn(0x7f9e980b6d40 0x7f9e98148a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:59.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.229+0000 7f9e9d9c9640 1 -- 172.21.15.5:0/1871938785 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e980b5e20 con 0x7f9e980b4c70 2024-03-18T15:33:59.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.233+0000 7f9e9d9c9640 1 --2- 172.21.15.5:0/1871938785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e980b4c70 0x7f9e98071380 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9e980725f0 tx=0x7f9e9001c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=835f4240387238cc server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:59.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.233+0000 7f9e8ffff640 1 -- 172.21.15.5:0/1871938785 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e900a9430 con 0x7f9e980b4c70 2024-03-18T15:33:59.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.233+0000 7f9e8ffff640 1 -- 172.21.15.5:0/1871938785 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e900ae070 con 0x7f9e980b4c70 2024-03-18T15:33:59.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.233+0000 7f9e8ffff640 1 -- 172.21.15.5:0/1871938785 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e900a97f0 con 0x7f9e980b4c70 2024-03-18T15:33:59.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.233+0000 7f9e9f42c640 1 -- 172.21.15.5:0/1871938785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e980b4c70 msgr2=0x7f9e98071380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:59.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.233+0000 7f9e9f42c640 1 --2- 172.21.15.5:0/1871938785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e980b4c70 0x7f9e98071380 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9e980725f0 tx=0x7f9e9001c6a0 comp rx=0 tx=0).stop 2024-03-18T15:33:59.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.233+0000 7f9e9f42c640 1 -- 172.21.15.5:0/1871938785 shutdown_connections 2024-03-18T15:33:59.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.233+0000 7f9e9f42c640 1 --2- 172.21.15.5:0/1871938785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e980b4c70 0x7f9e98071380 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:59.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.233+0000 7f9e9f42c640 1 --2- 172.21.15.5:0/1871938785 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9e98148f90 0x7f9e980b4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:59.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.233+0000 7f9e9f42c640 1 --2- 172.21.15.5:0/1871938785 >> v2:172.21.15.80:3300/0 conn(0x7f9e980b6d40 0x7f9e98148a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:59.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.233+0000 7f9e9f42c640 1 -- 172.21.15.5:0/1871938785 >> 172.21.15.5:0/1871938785 conn(0x7f9e98066fc0 msgr2=0x7f9e980673d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:59.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.233+0000 7f9e9f42c640 1 -- 172.21.15.5:0/1871938785 shutdown_connections 2024-03-18T15:33:59.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.233+0000 7f9e9f42c640 1 -- 172.21.15.5:0/1871938785 wait complete. 2024-03-18T15:33:59.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.233+0000 7f9e9f42c640 1 Processor -- start 2024-03-18T15:33:59.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.233+0000 7f9e9f42c640 1 -- start start 2024-03-18T15:33:59.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.233+0000 7f9e9f42c640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9e980b4c70 0x7f9e9806dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:59.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.233+0000 7f9e9f42c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e980b6d40 0x7f9e981401f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:59.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.233+0000 7f9e9f42c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e98148f90 0x7f9e98140730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:59.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.233+0000 7f9e9f42c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9e98151e10 con 0x7f9e98148f90 2024-03-18T15:33:59.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.233+0000 7f9e9f42c640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9e98151c90 con 0x7f9e980b6d40 2024-03-18T15:33:59.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.233+0000 7f9e9f42c640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f9e98151f90 con 0x7f9e980b4c70 2024-03-18T15:33:59.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.233+0000 7f9e9d9c9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e98148f90 0x7f9e98140730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:59.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.233+0000 7f9e9d9c9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e98148f90 0x7f9e98140730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43178/0 (socket says 172.21.15.5:43178) 2024-03-18T15:33:59.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.233+0000 7f9e9d9c9640 1 -- 172.21.15.5:0/2557286932 learned_addr learned my addr 172.21.15.5:0/2557286932 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:59.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.233+0000 7f9e9d1c8640 1 --2- 172.21.15.5:0/2557286932 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9e980b4c70 0x7f9e9806dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:59.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.233+0000 7f9e9c9c7640 1 --2- 172.21.15.5:0/2557286932 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e980b6d40 0x7f9e981401f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:59.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.233+0000 7f9e9d9c9640 1 -- 172.21.15.5:0/2557286932 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9e980b4c70 msgr2=0x7f9e9806dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:59.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.233+0000 7f9e9d9c9640 1 --2- 172.21.15.5:0/2557286932 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9e980b4c70 0x7f9e9806dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:59.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.233+0000 7f9e9d9c9640 1 -- 172.21.15.5:0/2557286932 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e980b6d40 msgr2=0x7f9e981401f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:59.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.233+0000 7f9e9d9c9640 1 --2- 172.21.15.5:0/2557286932 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e980b6d40 0x7f9e981401f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:59.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.233+0000 7f9e9d9c9640 1 -- 172.21.15.5:0/2557286932 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e98140e10 con 0x7f9e98148f90 2024-03-18T15:33:59.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.233+0000 7f9e9c9c7640 1 --2- 172.21.15.5:0/2557286932 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e980b6d40 0x7f9e981401f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:33:59.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.233+0000 7f9e9d1c8640 1 --2- 172.21.15.5:0/2557286932 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9e980b4c70 0x7f9e9806dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:33:59.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.233+0000 7f9e9d9c9640 1 --2- 172.21.15.5:0/2557286932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e98148f90 0x7f9e98140730 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9e9001cf40 tx=0x7f9e90019440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:59.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.233+0000 7f9e8dffb640 1 -- 172.21.15.5:0/2557286932 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e900b8070 con 0x7f9e98148f90 2024-03-18T15:33:59.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.233+0000 7f9e8dffb640 1 -- 172.21.15.5:0/2557286932 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e900a9840 con 0x7f9e98148f90 2024-03-18T15:33:59.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.233+0000 7f9e9f42c640 1 -- 172.21.15.5:0/2557286932 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e981e3780 con 0x7f9e98148f90 2024-03-18T15:33:59.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.233+0000 7f9e8dffb640 1 -- 172.21.15.5:0/2557286932 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e900a9c30 con 0x7f9e98148f90 2024-03-18T15:33:59.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.233+0000 7f9e9f42c640 1 -- 172.21.15.5:0/2557286932 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e981e3c90 con 0x7f9e98148f90 2024-03-18T15:33:59.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.237+0000 7f9e9f42c640 1 -- 172.21.15.5:0/2557286932 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e9806e310 con 0x7f9e98148f90 2024-03-18T15:33:59.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.237+0000 7f9e8dffb640 1 -- 172.21.15.5:0/2557286932 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f9e900c5430 con 0x7f9e98148f90 2024-03-18T15:33:59.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.237+0000 7f9e8dffb640 1 --2- 172.21.15.5:0/2557286932 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f9e7c067690 0x7f9e7c069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:59.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.237+0000 7f9e8dffb640 1 -- 172.21.15.5:0/2557286932 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f9e9011fb40 con 0x7f9e98148f90 2024-03-18T15:33:59.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.237+0000 7f9e9d1c8640 1 --2- 172.21.15.5:0/2557286932 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f9e7c067690 0x7f9e7c069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:59.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.237+0000 7f9e9d1c8640 1 --2- 172.21.15.5:0/2557286932 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f9e7c067690 0x7f9e7c069b50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9e940097c0 tx=0x7f9e94008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:59.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.241+0000 7f9e8dffb640 1 -- 172.21.15.5:0/2557286932 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f9e9806e310 con 0x7f9e98148f90 2024-03-18T15:33:59.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.397+0000 7f9e9f42c640 1 -- 172.21.15.5:0/2557286932 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9e9814f540 con 0x7f9e98148f90 2024-03-18T15:33:59.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.401+0000 7f9e8dffb640 1 -- 172.21.15.5:0/2557286932 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16449 (secure 0 0 0) 0x7f9e900165e0 con 0x7f9e98148f90 2024-03-18T15:33:59.402 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:33:59.402 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":21,"fsid":"ef54fe03-5aa7-4eb2-9b1e-66c9cc345b94","created":"2024-03-18T15:33:18.086614+0000","modified":"2024-03-18T15:33:43.984050+0000","last_up_change":"2024-03-18T15:33:29.229911+0000","last_in_change":"2024-03-18T15:33:20.323840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:28.963186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T15:33:40.533646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T15:33:41.323519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df1c754b-af59-4d16-8919-fbb6386b3c48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6804","nonce":976264754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6808","nonce":976264754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6815","nonce":976264754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6811","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6813","nonce":976264754}]},"public_addr":"172.21.15.5:6804/976264754","cluster_addr":"172.21.15.5:6808/976264754","heartbeat_back_addr":"172.21.15.5:6815/976264754","heartbeat_front_addr":"172.21.15.5:6813/976264754","state":["exists","up"]},{"osd":1,"uuid":"b9a08b5e-8af4-4144-bdba-d69ed1ac627a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6801","nonce":3901761126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6803","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6805","nonce":3901761126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6812","nonce":3901761126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6807","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6809","nonce":3901761126}]},"public_addr":"172.21.15.5:6801/3901761126","cluster_addr":"172.21.15.5:6805/3901761126","heartbeat_back_addr":"172.21.15.5:6812/3901761126","heartbeat_front_addr":"172.21.15.5:6809/3901761126","state":["exists","up"]},{"osd":2,"uuid":"c3e8bb76-e382-439d-b380-62ef92f1255a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6817","nonce":502968448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6819","nonce":502968448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6823","nonce":502968448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6821","nonce":502968448}]},"public_addr":"172.21.15.5:6817/502968448","cluster_addr":"172.21.15.5:6819/502968448","heartbeat_back_addr":"172.21.15.5:6823/502968448","heartbeat_front_addr":"172.21.15.5:6821/502968448","state":["exists","up"]},{"osd":3,"uuid":"e8c94625-f2cd-4eba-b53d-2e7356e1c02a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6825","nonce":2425932735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6827","nonce":2425932735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6831","nonce":2425932735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6829","nonce":2425932735}]},"public_addr":"172.21.15.5:6825/2425932735","cluster_addr":"172.21.15.5:6827/2425932735","heartbeat_back_addr":"172.21.15.5:6831/2425932735","heartbeat_front_addr":"172.21.15.5:6829/2425932735","state":["exists","up"]},{"osd":4,"uuid":"0099dd52-6265-4d32-82e4-3ef16e73add0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6817","nonce":976514285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6819","nonce":976514285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6823","nonce":976514285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6821","nonce":976514285}]},"public_addr":"172.21.15.80:6817/976514285","cluster_addr":"172.21.15.80:6819/976514285","heartbeat_back_addr":"172.21.15.80:6823/976514285","heartbeat_front_addr":"172.21.15.80:6821/976514285","state":["exists","up"]},{"osd":5,"uuid":"ed10c84f-8c39-41cc-9b64-7c482a94a48e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6825","nonce":627153447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6827","nonce":627153447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6831","nonce":627153447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6829","nonce":627153447}]},"public_addr":"172.21.15.80:6825/627153447","cluster_addr":"172.21.15.80:6827/627153447","heartbeat_back_addr":"172.21.15.80:6831/627153447","heartbeat_front_addr":"172.21.15.80:6829/627153447","state":["exists","up"]},{"osd":6,"uuid":"c7a3538a-7048-4fba-8781-b440f03c25db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6803","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6805","nonce":488342527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6807","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6809","nonce":488342527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6815","nonce":488342527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6811","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6813","nonce":488342527}]},"public_addr":"172.21.15.80:6805/488342527","cluster_addr":"172.21.15.80:6809/488342527","heartbeat_back_addr":"172.21.15.80:6815/488342527","heartbeat_front_addr":"172.21.15.80:6813/488342527","state":["exists","up"]},{"osd":7,"uuid":"f9418104-3a97-4110-bc3a-1f8151d0f757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6801","nonce":982831666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6804","nonce":982831666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6812","nonce":982831666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6808","nonce":982831666}]},"public_addr":"172.21.15.80:6801/982831666","cluster_addr":"172.21.15.80:6804/982831666","heartbeat_back_addr":"172.21.15.80:6812/982831666","heartbeat_front_addr":"172.21.15.80:6808/982831666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.970828+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.858116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.987556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.864603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.756293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.591502+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:33:59.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.401+0000 7f9e9f42c640 1 -- 172.21.15.5:0/2557286932 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f9e7c067690 msgr2=0x7f9e7c069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:59.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.401+0000 7f9e9f42c640 1 --2- 172.21.15.5:0/2557286932 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f9e7c067690 0x7f9e7c069b50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9e940097c0 tx=0x7f9e94008040 comp rx=0 tx=0).stop 2024-03-18T15:33:59.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.401+0000 7f9e9f42c640 1 -- 172.21.15.5:0/2557286932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e98148f90 msgr2=0x7f9e98140730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:59.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.401+0000 7f9e9f42c640 1 --2- 172.21.15.5:0/2557286932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e98148f90 0x7f9e98140730 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9e9001cf40 tx=0x7f9e90019440 comp rx=0 tx=0).stop 2024-03-18T15:33:59.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.401+0000 7f9e9f42c640 1 -- 172.21.15.5:0/2557286932 shutdown_connections 2024-03-18T15:33:59.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.401+0000 7f9e9f42c640 1 --2- 172.21.15.5:0/2557286932 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f9e7c067690 0x7f9e7c069b50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:59.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.401+0000 7f9e9f42c640 1 --2- 172.21.15.5:0/2557286932 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e98148f90 0x7f9e98140730 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:59.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.401+0000 7f9e9f42c640 1 --2- 172.21.15.5:0/2557286932 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e980b6d40 0x7f9e981401f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:59.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.401+0000 7f9e9f42c640 1 --2- 172.21.15.5:0/2557286932 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f9e980b4c70 0x7f9e9806dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:59.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.401+0000 7f9e9f42c640 1 -- 172.21.15.5:0/2557286932 >> 172.21.15.5:0/2557286932 conn(0x7f9e98066fc0 msgr2=0x7f9e980b3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:59.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.401+0000 7f9e9f42c640 1 -- 172.21.15.5:0/2557286932 shutdown_connections 2024-03-18T15:33:59.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.401+0000 7f9e9f42c640 1 -- 172.21.15.5:0/2557286932 wait complete. 2024-03-18T15:33:59.419 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T15:33:28.963186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-18T15:33:40.533646+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-18T15:33:41.323519+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T15:33:59.420 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T15:33:59.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.533+0000 7f051a2d5640 1 Processor -- start 2024-03-18T15:33:59.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.533+0000 7f051a2d5640 1 -- start start 2024-03-18T15:33:59.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.533+0000 7f051a2d5640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f05140b5710 0x7f051414a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:59.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.533+0000 7f051a2d5640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f051414ae60 0x7f0514140ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:59.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.533+0000 7f051a2d5640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05140b9d50 con 0x7f0514141430 2024-03-18T15:33:59.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.533+0000 7f051a2d5640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f05140b4320 con 0x7f051414ae60 2024-03-18T15:33:59.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.533+0000 7f051a2d5640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f05140b4570 con 0x7f05140b5710 2024-03-18T15:33:59.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.533+0000 7f05137fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f051414ae60 0x7f0514140ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:59.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.533+0000 7f05137fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f051414ae60 0x7f0514140ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43182/0 (socket says 172.21.15.5:43182) 2024-03-18T15:33:59.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.533+0000 7f05137fe640 1 -- 172.21.15.5:0/3531613811 learned_addr learned my addr 172.21.15.5:0/3531613811 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:59.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.533+0000 7f0513fff640 1 --2- 172.21.15.5:0/3531613811 >> v2:172.21.15.80:3300/0 conn(0x7f05140b5710 0x7f051414a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:59.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.533+0000 7f05137fe640 1 -- 172.21.15.5:0/3531613811 >> v2:172.21.15.80:3300/0 conn(0x7f05140b5710 msgr2=0x7f051414a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:59.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.533+0000 7f05137fe640 1 --2- 172.21.15.5:0/3531613811 >> v2:172.21.15.80:3300/0 conn(0x7f05140b5710 0x7f051414a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:59.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.533+0000 7f05137fe640 1 -- 172.21.15.5:0/3531613811 >> v1:172.21.15.80:6789/0 conn(0x7f0514141430 legacy=0x7f05140b7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:59.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.533+0000 7f05137fe640 1 -- 172.21.15.5:0/3531613811 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05140b4c30 con 0x7f051414ae60 2024-03-18T15:33:59.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.537+0000 7f05137fe640 1 --2- 172.21.15.5:0/3531613811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f051414ae60 0x7f0514140ef0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f051406c3e0 tx=0x7f050801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10f386fb84780a3d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:59.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.537+0000 7f0512ffd640 1 -- 172.21.15.5:0/3531613811 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05080b0070 con 0x7f051414ae60 2024-03-18T15:33:59.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.537+0000 7f0512ffd640 1 -- 172.21.15.5:0/3531613811 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05080ab600 con 0x7f051414ae60 2024-03-18T15:33:59.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.537+0000 7f0512ffd640 1 -- 172.21.15.5:0/3531613811 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05080ab9d0 con 0x7f051414ae60 2024-03-18T15:33:59.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.537+0000 7f051a2d5640 1 -- 172.21.15.5:0/3531613811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f051414ae60 msgr2=0x7f0514140ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:59.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.537+0000 7f051a2d5640 1 --2- 172.21.15.5:0/3531613811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f051414ae60 0x7f0514140ef0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f051406c3e0 tx=0x7f050801c3c0 comp rx=0 tx=0).stop 2024-03-18T15:33:59.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.537+0000 7f051a2d5640 1 -- 172.21.15.5:0/3531613811 shutdown_connections 2024-03-18T15:33:59.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.537+0000 7f051a2d5640 1 --2- 172.21.15.5:0/3531613811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f051414ae60 0x7f0514140ef0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:59.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.537+0000 7f051a2d5640 1 --2- 172.21.15.5:0/3531613811 >> v2:172.21.15.80:3300/0 conn(0x7f05140b5710 0x7f051414a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:59.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.537+0000 7f051a2d5640 1 -- 172.21.15.5:0/3531613811 >> 172.21.15.5:0/3531613811 conn(0x7f05140673c0 msgr2=0x7f05140677d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:59.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.537+0000 7f051a2d5640 1 -- 172.21.15.5:0/3531613811 shutdown_connections 2024-03-18T15:33:59.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.537+0000 7f051a2d5640 1 -- 172.21.15.5:0/3531613811 wait complete. 2024-03-18T15:33:59.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.537+0000 7f051a2d5640 1 Processor -- start 2024-03-18T15:33:59.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.537+0000 7f051a2d5640 1 -- start start 2024-03-18T15:33:59.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.537+0000 7f051a2d5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05140b5710 0x7f051413bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:59.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.537+0000 7f051a2d5640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0514141430 0x7f051413a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:59.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.537+0000 7f051a2d5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f051414ae60 0x7f05141e3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:59.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.537+0000 7f051a2d5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0514141880 con 0x7f05140b5710 2024-03-18T15:33:59.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.537+0000 7f051a2d5640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f051414b2c0 con 0x7f051414ae60 2024-03-18T15:33:59.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.537+0000 7f051a2d5640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f0514141a00 con 0x7f0514141430 2024-03-18T15:33:59.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.537+0000 7f0513fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05140b5710 0x7f051413bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:59.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.537+0000 7f0513fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05140b5710 0x7f051413bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43188/0 (socket says 172.21.15.5:43188) 2024-03-18T15:33:59.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.537+0000 7f0513fff640 1 -- 172.21.15.5:0/2614401649 learned_addr learned my addr 172.21.15.5:0/2614401649 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:59.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.537+0000 7f0518872640 1 --2- 172.21.15.5:0/2614401649 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f051414ae60 0x7f05141e3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:59.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.537+0000 7f05137fe640 1 --2- 172.21.15.5:0/2614401649 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0514141430 0x7f051413a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:59.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.537+0000 7f0513fff640 1 -- 172.21.15.5:0/2614401649 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0514141430 msgr2=0x7f051413a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:59.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.537+0000 7f0513fff640 1 --2- 172.21.15.5:0/2614401649 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0514141430 0x7f051413a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:59.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.537+0000 7f0513fff640 1 -- 172.21.15.5:0/2614401649 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f051414ae60 msgr2=0x7f05141e3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:59.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.537+0000 7f0513fff640 1 --2- 172.21.15.5:0/2614401649 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f051414ae60 0x7f05141e3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:59.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.537+0000 7f0513fff640 1 -- 172.21.15.5:0/2614401649 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0514142970 con 0x7f05140b5710 2024-03-18T15:33:59.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.537+0000 7f0513fff640 1 --2- 172.21.15.5:0/2614401649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05140b5710 0x7f051413bd40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f050400c7f0 tx=0x7f050400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:59.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.541+0000 7f0510ff9640 1 -- 172.21.15.5:0/2614401649 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0504015070 con 0x7f05140b5710 2024-03-18T15:33:59.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.541+0000 7f0510ff9640 1 -- 172.21.15.5:0/2614401649 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0504004240 con 0x7f05140b5710 2024-03-18T15:33:59.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.541+0000 7f051a2d5640 1 -- 172.21.15.5:0/2614401649 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f051413ae40 con 0x7f05140b5710 2024-03-18T15:33:59.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.541+0000 7f0510ff9640 1 -- 172.21.15.5:0/2614401649 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0504003620 con 0x7f05140b5710 2024-03-18T15:33:59.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.541+0000 7f051a2d5640 1 -- 172.21.15.5:0/2614401649 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f051413c280 con 0x7f05140b5710 2024-03-18T15:33:59.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.541+0000 7f051a2d5640 1 -- 172.21.15.5:0/2614401649 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f051404f550 con 0x7f05140b5710 2024-03-18T15:33:59.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.541+0000 7f0510ff9640 1 -- 172.21.15.5:0/2614401649 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f05040039d0 con 0x7f05140b5710 2024-03-18T15:33:59.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.541+0000 7f0510ff9640 1 --2- 172.21.15.5:0/2614401649 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0500067660 0x7f0500069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:59.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.541+0000 7f05137fe640 1 --2- 172.21.15.5:0/2614401649 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0500067660 0x7f0500069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:59.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.541+0000 7f0510ff9640 1 -- 172.21.15.5:0/2614401649 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f05040858c0 con 0x7f05140b5710 2024-03-18T15:33:59.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.541+0000 7f05137fe640 1 --2- 172.21.15.5:0/2614401649 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0500067660 0x7f0500069b20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f05080ad000 tx=0x7f05080a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:59.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.545+0000 7f0510ff9640 1 -- 172.21.15.5:0/2614401649 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f0504056530 con 0x7f05140b5710 2024-03-18T15:33:59.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.701+0000 7f051a2d5640 1 -- 172.21.15.5:0/2614401649 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f05140b7140 con 0x7f05140b5710 2024-03-18T15:33:59.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.701+0000 7f0510ff9640 1 -- 172.21.15.5:0/2614401649 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f0504055e30 con 0x7f05140b5710 2024-03-18T15:33:59.705 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-03-18T15:33:59.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.705+0000 7f051a2d5640 1 -- 172.21.15.5:0/2614401649 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0500067660 msgr2=0x7f0500069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:59.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.705+0000 7f051a2d5640 1 --2- 172.21.15.5:0/2614401649 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0500067660 0x7f0500069b20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f05080ad000 tx=0x7f05080a6000 comp rx=0 tx=0).stop 2024-03-18T15:33:59.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.705+0000 7f051a2d5640 1 -- 172.21.15.5:0/2614401649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05140b5710 msgr2=0x7f051413bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:59.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.705+0000 7f051a2d5640 1 --2- 172.21.15.5:0/2614401649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05140b5710 0x7f051413bd40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f050400c7f0 tx=0x7f050400ccb0 comp rx=0 tx=0).stop 2024-03-18T15:33:59.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.705+0000 7f051a2d5640 1 -- 172.21.15.5:0/2614401649 shutdown_connections 2024-03-18T15:33:59.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.705+0000 7f051a2d5640 1 --2- 172.21.15.5:0/2614401649 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0500067660 0x7f0500069b20 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:59.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.705+0000 7f051a2d5640 1 --2- 172.21.15.5:0/2614401649 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f051414ae60 0x7f05141e3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:59.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.705+0000 7f051a2d5640 1 --2- 172.21.15.5:0/2614401649 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0514141430 0x7f051413a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:59.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.705+0000 7f051a2d5640 1 --2- 172.21.15.5:0/2614401649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05140b5710 0x7f051413bd40 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:59.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.705+0000 7f051a2d5640 1 -- 172.21.15.5:0/2614401649 >> 172.21.15.5:0/2614401649 conn(0x7f05140673c0 msgr2=0x7f051413ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:59.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.705+0000 7f051a2d5640 1 -- 172.21.15.5:0/2614401649 shutdown_connections 2024-03-18T15:33:59.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.705+0000 7f051a2d5640 1 -- 172.21.15.5:0/2614401649 wait complete. 2024-03-18T15:33:59.722 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-18T15:33:59.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.833+0000 7fba7f4a0640 1 Processor -- start 2024-03-18T15:33:59.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.833+0000 7fba7f4a0640 1 -- start start 2024-03-18T15:33:59.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.837+0000 7fba7f4a0640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fba780aec00 0x7fba780aefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:59.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.837+0000 7fba7f4a0640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fba78143bd0 0x7fba78143ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:59.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.837+0000 7fba7f4a0640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba780add60 con 0x7fba78147060 2024-03-18T15:33:59.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.837+0000 7fba7f4a0640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fba780ae090 con 0x7fba78143bd0 2024-03-18T15:33:59.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.837+0000 7fba7f4a0640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fba780b0350 con 0x7fba780aec00 2024-03-18T15:33:59.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.837+0000 7fba7da3d640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fba78147060 0x7fba781474d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:46958/0 (socket says 172.21.15.5:46958) 2024-03-18T15:33:59.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.837+0000 7fba7da3d640 1 -- 172.21.15.5:0/1602677576 learned_addr learned my addr 172.21.15.5:0/1602677576 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:33:59.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.837+0000 7fba7d23c640 1 --2- 172.21.15.5:0/1602677576 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fba780aec00 0x7fba780aefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:59.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.837+0000 7fba7ca3b640 1 --2- 172.21.15.5:0/1602677576 >> v2:172.21.15.80:3300/0 conn(0x7fba78143bd0 0x7fba78143ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:59.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.837+0000 7fba77fff640 1 -- 172.21.15.5:0/1602677576 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1275652701 0 0) 0x7fba780add60 con 0x7fba78147060 2024-03-18T15:33:59.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.837+0000 7fba77fff640 1 -- 172.21.15.5:0/1602677576 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fba60003530 con 0x7fba78147060 2024-03-18T15:33:59.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.837+0000 7fba77fff640 1 -- 172.21.15.5:0/1602677576 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1139284764 0 0) 0x7fba60003530 con 0x7fba78147060 2024-03-18T15:33:59.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.837+0000 7fba77fff640 1 -- 172.21.15.5:0/1602677576 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fba780aec00 msgr2=0x7fba780aefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:59.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.837+0000 7fba77fff640 1 --2- 172.21.15.5:0/1602677576 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fba780aec00 0x7fba780aefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:59.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.837+0000 7fba77fff640 1 -- 172.21.15.5:0/1602677576 >> v2:172.21.15.80:3300/0 conn(0x7fba78143bd0 msgr2=0x7fba78143ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:59.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.837+0000 7fba77fff640 1 --2- 172.21.15.5:0/1602677576 >> v2:172.21.15.80:3300/0 conn(0x7fba78143bd0 0x7fba78143ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:59.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.837+0000 7fba77fff640 1 -- 172.21.15.5:0/1602677576 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba780b0630 con 0x7fba78147060 2024-03-18T15:33:59.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.837+0000 7fba77fff640 1 -- 172.21.15.5:0/1602677576 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3837549003 0 0) 0x7fba68001680 con 0x7fba78147060 2024-03-18T15:33:59.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.837+0000 7fba77fff640 1 -- 172.21.15.5:0/1602677576 >> v1:172.21.15.5:6789/0 conn(0x7fba78147060 legacy=0x7fba781474d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:59.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.837+0000 7fba77fff640 1 --2- 172.21.15.5:0/1602677576 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fba780aec00 0x7fba60003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:59.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.837+0000 7fba77fff640 1 --2- 172.21.15.5:0/1602677576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba60004020 0x7fba60004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:59.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.837+0000 7fba77fff640 1 --2- 172.21.15.5:0/1602677576 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba60004d50 0x7fba600051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:59.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.837+0000 7fba77fff640 1 -- 172.21.15.5:0/1602677576 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fba780ae090 con 0x7fba60004020 2024-03-18T15:33:59.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.837+0000 7fba77fff640 1 -- 172.21.15.5:0/1602677576 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fba780b0350 con 0x7fba60004d50 2024-03-18T15:33:59.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.837+0000 7fba77fff640 1 -- 172.21.15.5:0/1602677576 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fba6000ba50 con 0x7fba780aec00 2024-03-18T15:33:59.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.837+0000 7fba7ca3b640 1 --2- 172.21.15.5:0/1602677576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba60004020 0x7fba60004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:59.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.837+0000 7fba7d23c640 1 --2- 172.21.15.5:0/1602677576 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fba780aec00 0x7fba60003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:59.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.837+0000 7fba7da3d640 1 --2- 172.21.15.5:0/1602677576 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba60004d50 0x7fba600051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:59.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.837+0000 7fba7ca3b640 1 -- 172.21.15.5:0/1602677576 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fba780aec00 msgr2=0x7fba60003aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:59.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.837+0000 7fba7ca3b640 1 --2- 172.21.15.5:0/1602677576 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fba780aec00 0x7fba60003aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:59.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.837+0000 7fba7ca3b640 1 -- 172.21.15.5:0/1602677576 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba60004d50 msgr2=0x7fba600051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:33:59.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.837+0000 7fba7ca3b640 1 --2- 172.21.15.5:0/1602677576 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba60004d50 0x7fba600051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:59.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.837+0000 7fba7ca3b640 1 -- 172.21.15.5:0/1602677576 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba60003530 con 0x7fba60004020 2024-03-18T15:33:59.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.841+0000 7fba7ca3b640 1 --2- 172.21.15.5:0/1602677576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba60004020 0x7fba60004810 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fba6c009ae0 tx=0x7fba6c01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7675a2d070538386 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:59.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.841+0000 7fba77fff640 1 -- 172.21.15.5:0/1602677576 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c0b2070 con 0x7fba60004020 2024-03-18T15:33:59.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.841+0000 7fba77fff640 1 -- 172.21.15.5:0/1602677576 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba6c0a56a0 con 0x7fba60004020 2024-03-18T15:33:59.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.841+0000 7fba77fff640 1 -- 172.21.15.5:0/1602677576 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c0096b0 con 0x7fba60004020 2024-03-18T15:33:59.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.841+0000 7fba7f4a0640 1 -- 172.21.15.5:0/1602677576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba60004020 msgr2=0x7fba60004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:59.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.841+0000 7fba7f4a0640 1 --2- 172.21.15.5:0/1602677576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba60004020 0x7fba60004810 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fba6c009ae0 tx=0x7fba6c01cab0 comp rx=0 tx=0).stop 2024-03-18T15:33:59.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.841+0000 7fba7da3d640 1 -- 172.21.15.5:0/1602677576 reap_dead start 2024-03-18T15:33:59.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.841+0000 7fba7f4a0640 1 -- 172.21.15.5:0/1602677576 shutdown_connections 2024-03-18T15:33:59.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.841+0000 7fba7f4a0640 1 -- 172.21.15.5:0/1602677576 >> 172.21.15.5:0/1602677576 conn(0x7fba780b5670 msgr2=0x7fba780b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:33:59.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.841+0000 7fba7f4a0640 1 -- 172.21.15.5:0/1602677576 shutdown_connections 2024-03-18T15:33:59.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.841+0000 7fba7f4a0640 1 -- 172.21.15.5:0/1602677576 wait complete. 2024-03-18T15:33:59.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.841+0000 7fba7f4a0640 1 Processor -- start 2024-03-18T15:33:59.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.841+0000 7fba7f4a0640 1 -- start start 2024-03-18T15:33:59.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.841+0000 7fba7f4a0640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fba780b2090 0x7fba780b2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:59.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.841+0000 7fba7f4a0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba78145f40 0x7fba78146360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:59.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.841+0000 7fba7f4a0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba781dfef0 0x7fba781e03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:59.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.841+0000 7fba7f4a0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fba780ae4f0 con 0x7fba781dfef0 2024-03-18T15:33:59.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.841+0000 7fba7f4a0640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fba780b51c0 con 0x7fba78145f40 2024-03-18T15:33:59.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.841+0000 7fba7f4a0640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fba78147be0 con 0x7fba780b2090 2024-03-18T15:33:59.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.841+0000 7fba7da3d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba781dfef0 0x7fba781e03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:59.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.841+0000 7fba7da3d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba781dfef0 0x7fba781e03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43200/0 (socket says 172.21.15.5:43200) 2024-03-18T15:33:59.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.841+0000 7fba7da3d640 1 -- 172.21.15.5:0/3505083370 learned_addr learned my addr 172.21.15.5:0/3505083370 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:33:59.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.841+0000 7fba7ca3b640 1 --2- 172.21.15.5:0/3505083370 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba78145f40 0x7fba78146360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:59.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.841+0000 7fba7d23c640 1 --2- 172.21.15.5:0/3505083370 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fba780b2090 0x7fba780b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:59.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.841+0000 7fba7da3d640 1 -- 172.21.15.5:0/3505083370 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fba780b2090 msgr2=0x7fba780b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:59.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.841+0000 7fba7da3d640 1 --2- 172.21.15.5:0/3505083370 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fba780b2090 0x7fba780b2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:59.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.841+0000 7fba7da3d640 1 -- 172.21.15.5:0/3505083370 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba78145f40 msgr2=0x7fba78146360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:33:59.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.841+0000 7fba7da3d640 1 --2- 172.21.15.5:0/3505083370 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba78145f40 0x7fba78146360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:33:59.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.841+0000 7fba7da3d640 1 -- 172.21.15.5:0/3505083370 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba78145980 con 0x7fba781dfef0 2024-03-18T15:33:59.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.841+0000 7fba7d23c640 1 --2- 172.21.15.5:0/3505083370 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fba780b2090 0x7fba780b2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:33:59.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.841+0000 7fba7da3d640 1 --2- 172.21.15.5:0/3505083370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba781dfef0 0x7fba781e03a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fba7806dc10 tx=0x7fba68011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:59.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.841+0000 7fba75ffb640 1 -- 172.21.15.5:0/3505083370 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6801c070 con 0x7fba781dfef0 2024-03-18T15:33:59.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.841+0000 7fba75ffb640 1 -- 172.21.15.5:0/3505083370 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba680137a0 con 0x7fba781dfef0 2024-03-18T15:33:59.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.841+0000 7fba7f4a0640 1 -- 172.21.15.5:0/3505083370 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba78145c20 con 0x7fba781dfef0 2024-03-18T15:33:59.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.841+0000 7fba75ffb640 1 -- 172.21.15.5:0/3505083370 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba68010040 con 0x7fba781dfef0 2024-03-18T15:33:59.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.841+0000 7fba7f4a0640 1 -- 172.21.15.5:0/3505083370 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba7813c280 con 0x7fba781dfef0 2024-03-18T15:33:59.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.845+0000 7fba7f4a0640 1 -- 172.21.15.5:0/3505083370 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba7804f550 con 0x7fba781dfef0 2024-03-18T15:33:59.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.845+0000 7fba75ffb640 1 -- 172.21.15.5:0/3505083370 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fba6801f070 con 0x7fba781dfef0 2024-03-18T15:33:59.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.845+0000 7fba75ffb640 1 --2- 172.21.15.5:0/3505083370 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fba64067660 0x7fba64069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:33:59.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.845+0000 7fba7d23c640 1 --2- 172.21.15.5:0/3505083370 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fba64067660 0x7fba64069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:33:59.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.845+0000 7fba75ffb640 1 -- 172.21.15.5:0/3505083370 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fba68088190 con 0x7fba781dfef0 2024-03-18T15:33:59.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.845+0000 7fba7d23c640 1 --2- 172.21.15.5:0/3505083370 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fba64067660 0x7fba64069b20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fba70012430 tx=0x7fba7000b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:33:59.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:33:59.849+0000 7fba75ffb640 1 -- 172.21.15.5:0/3505083370 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fba68058e00 con 0x7fba781dfef0 2024-03-18T15:34:00.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.009+0000 7fba7f4a0640 1 -- 172.21.15.5:0/3505083370 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fba7806b990 con 0x7fba781dfef0 2024-03-18T15:34:00.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.009+0000 7fba75ffb640 1 -- 172.21.15.5:0/3505083370 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7fba68058700 con 0x7fba781dfef0 2024-03-18T15:34:00.011 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-03-18T15:34:00.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.013+0000 7fba7f4a0640 1 -- 172.21.15.5:0/3505083370 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fba64067660 msgr2=0x7fba64069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:00.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.013+0000 7fba7f4a0640 1 --2- 172.21.15.5:0/3505083370 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fba64067660 0x7fba64069b20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fba70012430 tx=0x7fba7000b040 comp rx=0 tx=0).stop 2024-03-18T15:34:00.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.013+0000 7fba7f4a0640 1 -- 172.21.15.5:0/3505083370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba781dfef0 msgr2=0x7fba781e03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:00.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.013+0000 7fba7f4a0640 1 --2- 172.21.15.5:0/3505083370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba781dfef0 0x7fba781e03a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fba7806dc10 tx=0x7fba68011920 comp rx=0 tx=0).stop 2024-03-18T15:34:00.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.013+0000 7fba7f4a0640 1 -- 172.21.15.5:0/3505083370 shutdown_connections 2024-03-18T15:34:00.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.013+0000 7fba7f4a0640 1 --2- 172.21.15.5:0/3505083370 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fba64067660 0x7fba64069b20 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:00.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.013+0000 7fba7f4a0640 1 --2- 172.21.15.5:0/3505083370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fba781dfef0 0x7fba781e03a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:00.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.013+0000 7fba7f4a0640 1 --2- 172.21.15.5:0/3505083370 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba78145f40 0x7fba78146360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:00.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.013+0000 7fba7f4a0640 1 --2- 172.21.15.5:0/3505083370 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fba780b2090 0x7fba780b2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:00.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.013+0000 7fba7f4a0640 1 -- 172.21.15.5:0/3505083370 >> 172.21.15.5:0/3505083370 conn(0x7fba780b5670 msgr2=0x7fba780b4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:00.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.013+0000 7fba7f4a0640 1 -- 172.21.15.5:0/3505083370 shutdown_connections 2024-03-18T15:34:00.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.013+0000 7fba7f4a0640 1 -- 172.21.15.5:0/3505083370 wait complete. 2024-03-18T15:34:00.030 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-18T15:34:00.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.149+0000 7f1ebcb2f640 1 Processor -- start 2024-03-18T15:34:00.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.149+0000 7f1ebcb2f640 1 -- start start 2024-03-18T15:34:00.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.149+0000 7f1ebcb2f640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1eb806b4e0 0x7f1eb80702b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:00.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.149+0000 7f1ebcb2f640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1eb8141360 0x7f1eb81417e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:00.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.149+0000 7f1ebcb2f640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1eb806b8c0 con 0x7f1eb81474c0 2024-03-18T15:34:00.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.149+0000 7f1ebcb2f640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f1eb806c0e0 con 0x7f1eb806b4e0 2024-03-18T15:34:00.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.149+0000 7f1ebcb2f640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f1eb80719b0 con 0x7f1eb8141360 2024-03-18T15:34:00.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.149+0000 7f1eb659c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1eb806b4e0 0x7f1eb80702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:00.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.149+0000 7f1eb659c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1eb806b4e0 0x7f1eb80702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43216/0 (socket says 172.21.15.5:43216) 2024-03-18T15:34:00.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.149+0000 7f1eb659c640 1 -- 172.21.15.5:0/4093755695 learned_addr learned my addr 172.21.15.5:0/4093755695 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:00.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.149+0000 7f1eb5d9b640 1 --2- 172.21.15.5:0/4093755695 >> v2:172.21.15.80:3300/0 conn(0x7f1eb8141360 0x7f1eb81417e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:00.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.149+0000 7f1eb659c640 1 -- 172.21.15.5:0/4093755695 >> v2:172.21.15.80:3300/0 conn(0x7f1eb8141360 msgr2=0x7f1eb81417e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:00.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.149+0000 7f1eb659c640 1 --2- 172.21.15.5:0/4093755695 >> v2:172.21.15.80:3300/0 conn(0x7f1eb8141360 0x7f1eb81417e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:00.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.149+0000 7f1eb659c640 1 -- 172.21.15.5:0/4093755695 >> v1:172.21.15.80:6789/0 conn(0x7f1eb81474c0 legacy=0x7f1eb814aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:00.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.149+0000 7f1eb659c640 1 -- 172.21.15.5:0/4093755695 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb806e800 con 0x7f1eb806b4e0 2024-03-18T15:34:00.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.153+0000 7f1eb659c640 1 --2- 172.21.15.5:0/4093755695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1eb806b4e0 0x7f1eb80702b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8073300 tx=0x7f1ea801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd8ff8e697ff4979 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:00.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.153+0000 7f1eb559a640 1 -- 172.21.15.5:0/4093755695 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea80b0070 con 0x7f1eb806b4e0 2024-03-18T15:34:00.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.153+0000 7f1eb559a640 1 -- 172.21.15.5:0/4093755695 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ea80ab600 con 0x7f1eb806b4e0 2024-03-18T15:34:00.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.153+0000 7f1eb559a640 1 -- 172.21.15.5:0/4093755695 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea80ab9d0 con 0x7f1eb806b4e0 2024-03-18T15:34:00.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.153+0000 7f1ebcb2f640 1 -- 172.21.15.5:0/4093755695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1eb806b4e0 msgr2=0x7f1eb80702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:00.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.153+0000 7f1ebcb2f640 1 --2- 172.21.15.5:0/4093755695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1eb806b4e0 0x7f1eb80702b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8073300 tx=0x7f1ea801c450 comp rx=0 tx=0).stop 2024-03-18T15:34:00.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.153+0000 7f1ebcb2f640 1 -- 172.21.15.5:0/4093755695 shutdown_connections 2024-03-18T15:34:00.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.153+0000 7f1ebcb2f640 1 --2- 172.21.15.5:0/4093755695 >> v2:172.21.15.80:3300/0 conn(0x7f1eb8141360 0x7f1eb81417e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:00.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.153+0000 7f1ebcb2f640 1 --2- 172.21.15.5:0/4093755695 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1eb806b4e0 0x7f1eb80702b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:00.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.153+0000 7f1ebcb2f640 1 -- 172.21.15.5:0/4093755695 >> 172.21.15.5:0/4093755695 conn(0x7f1eb80b7a80 msgr2=0x7f1eb80b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:00.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.153+0000 7f1ebcb2f640 1 -- 172.21.15.5:0/4093755695 shutdown_connections 2024-03-18T15:34:00.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.153+0000 7f1ebcb2f640 1 -- 172.21.15.5:0/4093755695 wait complete. 2024-03-18T15:34:00.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.153+0000 7f1ebcb2f640 1 Processor -- start 2024-03-18T15:34:00.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.153+0000 7f1ebcb2f640 1 -- start start 2024-03-18T15:34:00.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.153+0000 7f1ebcb2f640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1eb806b4e0 0x7f1eb81e3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:00.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.153+0000 7f1ebcb2f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1eb8141360 0x7f1eb80b3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:00.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.153+0000 7f1ebcb2f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1eb81474c0 0x7f1eb80b1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:00.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.153+0000 7f1ebcb2f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb8154f00 con 0x7f1eb81474c0 2024-03-18T15:34:00.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.153+0000 7f1ebcb2f640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb8154d80 con 0x7f1eb8141360 2024-03-18T15:34:00.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.153+0000 7f1ebcb2f640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f1eb8155080 con 0x7f1eb806b4e0 2024-03-18T15:34:00.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.153+0000 7f1eb6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1eb81474c0 0x7f1eb80b1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:00.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.153+0000 7f1eb6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1eb81474c0 0x7f1eb80b1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43222/0 (socket says 172.21.15.5:43222) 2024-03-18T15:34:00.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.153+0000 7f1eb6d9d640 1 -- 172.21.15.5:0/1902253660 learned_addr learned my addr 172.21.15.5:0/1902253660 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:00.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.153+0000 7f1eb659c640 1 --2- 172.21.15.5:0/1902253660 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1eb806b4e0 0x7f1eb81e3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:00.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.153+0000 7f1eb5d9b640 1 --2- 172.21.15.5:0/1902253660 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1eb8141360 0x7f1eb80b3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:00.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.157+0000 7f1eb6d9d640 1 -- 172.21.15.5:0/1902253660 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1eb806b4e0 msgr2=0x7f1eb81e3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:00.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.157+0000 7f1eb6d9d640 1 --2- 172.21.15.5:0/1902253660 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1eb806b4e0 0x7f1eb81e3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:00.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.157+0000 7f1eb6d9d640 1 -- 172.21.15.5:0/1902253660 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1eb8141360 msgr2=0x7f1eb80b3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:00.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.157+0000 7f1eb6d9d640 1 --2- 172.21.15.5:0/1902253660 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1eb8141360 0x7f1eb80b3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:00.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.157+0000 7f1eb6d9d640 1 -- 172.21.15.5:0/1902253660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb80b37a0 con 0x7f1eb81474c0 2024-03-18T15:34:00.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.157+0000 7f1eb6d9d640 1 --2- 172.21.15.5:0/1902253660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1eb81474c0 0x7f1eb80b1fd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000b9a0 tx=0x7f1eb000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:00.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.157+0000 7f1ea6ffd640 1 -- 172.21.15.5:0/1902253660 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb0019070 con 0x7f1eb81474c0 2024-03-18T15:34:00.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.157+0000 7f1ea6ffd640 1 -- 172.21.15.5:0/1902253660 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1eb000cba0 con 0x7f1eb81474c0 2024-03-18T15:34:00.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.157+0000 7f1ea6ffd640 1 -- 172.21.15.5:0/1902253660 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb0014440 con 0x7f1eb81474c0 2024-03-18T15:34:00.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.157+0000 7f1ebcb2f640 1 -- 172.21.15.5:0/1902253660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eb8144ed0 con 0x7f1eb81474c0 2024-03-18T15:34:00.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.157+0000 7f1ebcb2f640 1 -- 172.21.15.5:0/1902253660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eb80b12a0 con 0x7f1eb81474c0 2024-03-18T15:34:00.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.157+0000 7f1ebcb2f640 1 -- 172.21.15.5:0/1902253660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1eb804f550 con 0x7f1eb81474c0 2024-03-18T15:34:00.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.157+0000 7f1ea6ffd640 1 -- 172.21.15.5:0/1902253660 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f1eb001c070 con 0x7f1eb81474c0 2024-03-18T15:34:00.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.157+0000 7f1ea6ffd640 1 --2- 172.21.15.5:0/1902253660 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f1e98067660 0x7f1e98069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:00.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.157+0000 7f1ea6ffd640 1 -- 172.21.15.5:0/1902253660 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1eb0085520 con 0x7f1eb81474c0 2024-03-18T15:34:00.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.157+0000 7f1eb659c640 1 --2- 172.21.15.5:0/1902253660 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f1e98067660 0x7f1e98069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:00.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.157+0000 7f1eb659c640 1 --2- 172.21.15.5:0/1902253660 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f1e98067660 0x7f1e98069b20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1ea80ad000 tx=0x7f1ea8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:00.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.161+0000 7f1ea6ffd640 1 -- 172.21.15.5:0/1902253660 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f1eb0056190 con 0x7f1eb81474c0 2024-03-18T15:34:00.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.321+0000 7f1ebcb2f640 1 -- 172.21.15.5:0/1902253660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1eb80b9600 con 0x7f1eb81474c0 2024-03-18T15:34:00.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.321+0000 7f1ea6ffd640 1 -- 172.21.15.5:0/1902253660 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f1eb0055a90 con 0x7f1eb81474c0 2024-03-18T15:34:00.324 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-03-18T15:34:00.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.325+0000 7f1ebcb2f640 1 -- 172.21.15.5:0/1902253660 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f1e98067660 msgr2=0x7f1e98069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:00.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.325+0000 7f1ebcb2f640 1 --2- 172.21.15.5:0/1902253660 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f1e98067660 0x7f1e98069b20 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1ea80ad000 tx=0x7f1ea8008040 comp rx=0 tx=0).stop 2024-03-18T15:34:00.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.325+0000 7f1ebcb2f640 1 -- 172.21.15.5:0/1902253660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1eb81474c0 msgr2=0x7f1eb80b1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:00.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.325+0000 7f1ebcb2f640 1 --2- 172.21.15.5:0/1902253660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1eb81474c0 0x7f1eb80b1fd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000b9a0 tx=0x7f1eb000be60 comp rx=0 tx=0).stop 2024-03-18T15:34:00.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.325+0000 7f1ebcb2f640 1 -- 172.21.15.5:0/1902253660 shutdown_connections 2024-03-18T15:34:00.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.325+0000 7f1ebcb2f640 1 --2- 172.21.15.5:0/1902253660 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f1e98067660 0x7f1e98069b20 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:00.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.325+0000 7f1ebcb2f640 1 --2- 172.21.15.5:0/1902253660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1eb81474c0 0x7f1eb80b1fd0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:00.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.325+0000 7f1ebcb2f640 1 --2- 172.21.15.5:0/1902253660 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1eb8141360 0x7f1eb80b3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:00.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.325+0000 7f1ebcb2f640 1 --2- 172.21.15.5:0/1902253660 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1eb806b4e0 0x7f1eb81e3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:00.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.325+0000 7f1ebcb2f640 1 -- 172.21.15.5:0/1902253660 >> 172.21.15.5:0/1902253660 conn(0x7f1eb80b7a80 msgr2=0x7f1eb813d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:00.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.325+0000 7f1ebcb2f640 1 -- 172.21.15.5:0/1902253660 shutdown_connections 2024-03-18T15:34:00.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.325+0000 7f1ebcb2f640 1 -- 172.21.15.5:0/1902253660 wait complete. 2024-03-18T15:34:00.343 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:34:00.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.465+0000 7f528c850640 1 Processor -- start 2024-03-18T15:34:00.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.465+0000 7f528c850640 1 -- start start 2024-03-18T15:34:00.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.465+0000 7f528c850640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f528806d2d0 0x7f52880b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:00.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.465+0000 7f528c850640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f528813f260 0x7f528813f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:00.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.465+0000 7f528c850640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52880b0ea0 con 0x7f52881453a0 2024-03-18T15:34:00.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.465+0000 7f528c850640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f52880728c0 con 0x7f528813f260 2024-03-18T15:34:00.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.465+0000 7f528c850640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f528806d6b0 con 0x7f528806d2d0 2024-03-18T15:34:00.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.465+0000 7f5285d9b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f528813f260 0x7f528813f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:00.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.465+0000 7f5285d9b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f528813f260 0x7f528813f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43232/0 (socket says 172.21.15.5:43232) 2024-03-18T15:34:00.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.465+0000 7f5285d9b640 1 -- 172.21.15.5:0/115999623 learned_addr learned my addr 172.21.15.5:0/115999623 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:00.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.465+0000 7f528659c640 1 --2- 172.21.15.5:0/115999623 >> v2:172.21.15.80:3300/0 conn(0x7f528806d2d0 0x7f52880b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:00.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.465+0000 7f5285d9b640 1 -- 172.21.15.5:0/115999623 >> v2:172.21.15.80:3300/0 conn(0x7f528806d2d0 msgr2=0x7f52880b23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:00.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.465+0000 7f5285d9b640 1 --2- 172.21.15.5:0/115999623 >> v2:172.21.15.80:3300/0 conn(0x7f528806d2d0 0x7f52880b23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:00.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.465+0000 7f5285d9b640 1 -- 172.21.15.5:0/115999623 >> v1:172.21.15.80:6789/0 conn(0x7f52881453a0 legacy=0x7f52881457f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:00.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.465+0000 7f5285d9b640 1 -- 172.21.15.5:0/115999623 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52880b0bc0 con 0x7f528813f260 2024-03-18T15:34:00.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.469+0000 7f5285d9b640 1 --2- 172.21.15.5:0/115999623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f528813f260 0x7f528813f6a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f528806e9d0 tx=0x7f527801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e145e95ce4a38372 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:00.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.469+0000 7f528559a640 1 -- 172.21.15.5:0/115999623 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52780b0070 con 0x7f528813f260 2024-03-18T15:34:00.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.469+0000 7f528559a640 1 -- 172.21.15.5:0/115999623 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52780ab600 con 0x7f528813f260 2024-03-18T15:34:00.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.469+0000 7f528559a640 1 -- 172.21.15.5:0/115999623 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52780ab9d0 con 0x7f528813f260 2024-03-18T15:34:00.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.469+0000 7f528c850640 1 -- 172.21.15.5:0/115999623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f528813f260 msgr2=0x7f528813f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:00.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.469+0000 7f528c850640 1 --2- 172.21.15.5:0/115999623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f528813f260 0x7f528813f6a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f528806e9d0 tx=0x7f527801c490 comp rx=0 tx=0).stop 2024-03-18T15:34:00.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.469+0000 7f528c850640 1 -- 172.21.15.5:0/115999623 shutdown_connections 2024-03-18T15:34:00.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.469+0000 7f528c850640 1 --2- 172.21.15.5:0/115999623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f528813f260 0x7f528813f6a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:00.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.469+0000 7f528c850640 1 --2- 172.21.15.5:0/115999623 >> v2:172.21.15.80:3300/0 conn(0x7f528806d2d0 0x7f52880b23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:00.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.469+0000 7f528c850640 1 -- 172.21.15.5:0/115999623 >> 172.21.15.5:0/115999623 conn(0x7f52880b5670 msgr2=0x7f52880b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:00.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.469+0000 7f528c850640 1 -- 172.21.15.5:0/115999623 shutdown_connections 2024-03-18T15:34:00.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.469+0000 7f528c850640 1 -- 172.21.15.5:0/115999623 wait complete. 2024-03-18T15:34:00.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.469+0000 7f528c850640 1 Processor -- start 2024-03-18T15:34:00.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.469+0000 7f528c850640 1 -- start start 2024-03-18T15:34:00.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.469+0000 7f528c850640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f528806d2d0 0x7f5288140090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:00.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.469+0000 7f528c850640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f528813f260 0x7f5288141550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:00.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.469+0000 7f528c850640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f52881453a0 0x7f5288141d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:00.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.469+0000 7f528c850640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f52880b52c0 con 0x7f528813f260 2024-03-18T15:34:00.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.469+0000 7f528c850640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f52880b5140 con 0x7f52881453a0 2024-03-18T15:34:00.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.469+0000 7f528c850640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f5288072d30 con 0x7f528806d2d0 2024-03-18T15:34:00.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.469+0000 7f5285d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f528813f260 0x7f5288141550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:00.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.469+0000 7f5285d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f528813f260 0x7f5288141550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43240/0 (socket says 172.21.15.5:43240) 2024-03-18T15:34:00.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.469+0000 7f5285d9b640 1 -- 172.21.15.5:0/2051628924 learned_addr learned my addr 172.21.15.5:0/2051628924 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:00.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.469+0000 7f5286d9d640 1 --2- 172.21.15.5:0/2051628924 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f52881453a0 0x7f5288141d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:00.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.469+0000 7f528659c640 1 --2- 172.21.15.5:0/2051628924 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f528806d2d0 0x7f5288140090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:00.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.469+0000 7f5285d9b640 1 -- 172.21.15.5:0/2051628924 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f528806d2d0 msgr2=0x7f5288140090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:00.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.469+0000 7f5285d9b640 1 --2- 172.21.15.5:0/2051628924 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f528806d2d0 0x7f5288140090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:00.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.469+0000 7f5285d9b640 1 -- 172.21.15.5:0/2051628924 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f52881453a0 msgr2=0x7f5288141d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:00.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.469+0000 7f5285d9b640 1 --2- 172.21.15.5:0/2051628924 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f52881453a0 0x7f5288141d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:00.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.469+0000 7f5285d9b640 1 -- 172.21.15.5:0/2051628924 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52881428a0 con 0x7f528813f260 2024-03-18T15:34:00.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.469+0000 7f528659c640 1 --2- 172.21.15.5:0/2051628924 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f528806d2d0 0x7f5288140090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:00.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.469+0000 7f5285d9b640 1 --2- 172.21.15.5:0/2051628924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f528813f260 0x7f5288141550 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5278000c10 tx=0x7f5278019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:00.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.469+0000 7f5276ffd640 1 -- 172.21.15.5:0/2051628924 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52780b0040 con 0x7f528813f260 2024-03-18T15:34:00.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.469+0000 7f5276ffd640 1 -- 172.21.15.5:0/2051628924 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52780b9070 con 0x7f528813f260 2024-03-18T15:34:00.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.469+0000 7f528c850640 1 -- 172.21.15.5:0/2051628924 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5288072180 con 0x7f528813f260 2024-03-18T15:34:00.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.469+0000 7f5276ffd640 1 -- 172.21.15.5:0/2051628924 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52780b4650 con 0x7f528813f260 2024-03-18T15:34:00.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.469+0000 7f528c850640 1 -- 172.21.15.5:0/2051628924 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5288141a90 con 0x7f528813f260 2024-03-18T15:34:00.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.473+0000 7f528c850640 1 -- 172.21.15.5:0/2051628924 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5288141230 con 0x7f528813f260 2024-03-18T15:34:00.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.473+0000 7f5276ffd640 1 -- 172.21.15.5:0/2051628924 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f52780b47f0 con 0x7f528813f260 2024-03-18T15:34:00.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.473+0000 7f5276ffd640 1 --2- 172.21.15.5:0/2051628924 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5268067660 0x7f5268069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:00.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.473+0000 7f5276ffd640 1 -- 172.21.15.5:0/2051628924 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f527811f520 con 0x7f528813f260 2024-03-18T15:34:00.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.473+0000 7f528659c640 1 --2- 172.21.15.5:0/2051628924 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5268067660 0x7f5268069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:00.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.473+0000 7f528659c640 1 --2- 172.21.15.5:0/2051628924 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5268067660 0x7f5268069b20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f527c009ae0 tx=0x7f527c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:00.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.477+0000 7f5276ffd640 1 -- 172.21.15.5:0/2051628924 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f5288141230 con 0x7f528813f260 2024-03-18T15:34:00.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.633+0000 7f528c850640 1 -- 172.21.15.5:0/2051628924 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f528804f550 con 0x7f528813f260 2024-03-18T15:34:00.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.633+0000 7f5276ffd640 1 -- 172.21.15.5:0/2051628924 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16449 (secure 0 0 0) 0x7f5278016110 con 0x7f528813f260 2024-03-18T15:34:00.636 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:34:00.636 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":21,"fsid":"ef54fe03-5aa7-4eb2-9b1e-66c9cc345b94","created":"2024-03-18T15:33:18.086614+0000","modified":"2024-03-18T15:33:43.984050+0000","last_up_change":"2024-03-18T15:33:29.229911+0000","last_in_change":"2024-03-18T15:33:20.323840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:28.963186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T15:33:40.533646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T15:33:41.323519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df1c754b-af59-4d16-8919-fbb6386b3c48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6804","nonce":976264754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6808","nonce":976264754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6815","nonce":976264754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6811","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6813","nonce":976264754}]},"public_addr":"172.21.15.5:6804/976264754","cluster_addr":"172.21.15.5:6808/976264754","heartbeat_back_addr":"172.21.15.5:6815/976264754","heartbeat_front_addr":"172.21.15.5:6813/976264754","state":["exists","up"]},{"osd":1,"uuid":"b9a08b5e-8af4-4144-bdba-d69ed1ac627a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6801","nonce":3901761126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6803","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6805","nonce":3901761126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6812","nonce":3901761126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6807","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6809","nonce":3901761126}]},"public_addr":"172.21.15.5:6801/3901761126","cluster_addr":"172.21.15.5:6805/3901761126","heartbeat_back_addr":"172.21.15.5:6812/3901761126","heartbeat_front_addr":"172.21.15.5:6809/3901761126","state":["exists","up"]},{"osd":2,"uuid":"c3e8bb76-e382-439d-b380-62ef92f1255a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6817","nonce":502968448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6819","nonce":502968448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6823","nonce":502968448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6821","nonce":502968448}]},"public_addr":"172.21.15.5:6817/502968448","cluster_addr":"172.21.15.5:6819/502968448","heartbeat_back_addr":"172.21.15.5:6823/502968448","heartbeat_front_addr":"172.21.15.5:6821/502968448","state":["exists","up"]},{"osd":3,"uuid":"e8c94625-f2cd-4eba-b53d-2e7356e1c02a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6825","nonce":2425932735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6827","nonce":2425932735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6831","nonce":2425932735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6829","nonce":2425932735}]},"public_addr":"172.21.15.5:6825/2425932735","cluster_addr":"172.21.15.5:6827/2425932735","heartbeat_back_addr":"172.21.15.5:6831/2425932735","heartbeat_front_addr":"172.21.15.5:6829/2425932735","state":["exists","up"]},{"osd":4,"uuid":"0099dd52-6265-4d32-82e4-3ef16e73add0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6817","nonce":976514285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6819","nonce":976514285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6823","nonce":976514285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6821","nonce":976514285}]},"public_addr":"172.21.15.80:6817/976514285","cluster_addr":"172.21.15.80:6819/976514285","heartbeat_back_addr":"172.21.15.80:6823/976514285","heartbeat_front_addr":"172.21.15.80:6821/976514285","state":["exists","up"]},{"osd":5,"uuid":"ed10c84f-8c39-41cc-9b64-7c482a94a48e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6825","nonce":627153447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6827","nonce":627153447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6831","nonce":627153447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6829","nonce":627153447}]},"public_addr":"172.21.15.80:6825/627153447","cluster_addr":"172.21.15.80:6827/627153447","heartbeat_back_addr":"172.21.15.80:6831/627153447","heartbeat_front_addr":"172.21.15.80:6829/627153447","state":["exists","up"]},{"osd":6,"uuid":"c7a3538a-7048-4fba-8781-b440f03c25db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6803","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6805","nonce":488342527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6807","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6809","nonce":488342527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6815","nonce":488342527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6811","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6813","nonce":488342527}]},"public_addr":"172.21.15.80:6805/488342527","cluster_addr":"172.21.15.80:6809/488342527","heartbeat_back_addr":"172.21.15.80:6815/488342527","heartbeat_front_addr":"172.21.15.80:6813/488342527","state":["exists","up"]},{"osd":7,"uuid":"f9418104-3a97-4110-bc3a-1f8151d0f757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6801","nonce":982831666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6804","nonce":982831666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6812","nonce":982831666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6808","nonce":982831666}]},"public_addr":"172.21.15.80:6801/982831666","cluster_addr":"172.21.15.80:6804/982831666","heartbeat_back_addr":"172.21.15.80:6812/982831666","heartbeat_front_addr":"172.21.15.80:6808/982831666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.970828+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.858116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.987556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.864603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.756293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.591502+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:34:00.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.637+0000 7f528c850640 1 -- 172.21.15.5:0/2051628924 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5268067660 msgr2=0x7f5268069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:00.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.637+0000 7f528c850640 1 --2- 172.21.15.5:0/2051628924 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5268067660 0x7f5268069b20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f527c009ae0 tx=0x7f527c008040 comp rx=0 tx=0).stop 2024-03-18T15:34:00.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.637+0000 7f528c850640 1 -- 172.21.15.5:0/2051628924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f528813f260 msgr2=0x7f5288141550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:00.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.637+0000 7f528c850640 1 --2- 172.21.15.5:0/2051628924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f528813f260 0x7f5288141550 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5278000c10 tx=0x7f5278019230 comp rx=0 tx=0).stop 2024-03-18T15:34:00.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.637+0000 7f528c850640 1 -- 172.21.15.5:0/2051628924 shutdown_connections 2024-03-18T15:34:00.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.637+0000 7f528c850640 1 --2- 172.21.15.5:0/2051628924 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5268067660 0x7f5268069b20 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:00.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.637+0000 7f528c850640 1 --2- 172.21.15.5:0/2051628924 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f52881453a0 0x7f5288141d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:00.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.637+0000 7f528c850640 1 --2- 172.21.15.5:0/2051628924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f528813f260 0x7f5288141550 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:00.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.637+0000 7f528c850640 1 --2- 172.21.15.5:0/2051628924 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f528806d2d0 0x7f5288140090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:00.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.637+0000 7f528c850640 1 -- 172.21.15.5:0/2051628924 >> 172.21.15.5:0/2051628924 conn(0x7f52880b5670 msgr2=0x7f52880b7160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:00.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.637+0000 7f528c850640 1 -- 172.21.15.5:0/2051628924 shutdown_connections 2024-03-18T15:34:00.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.637+0000 7f528c850640 1 -- 172.21.15.5:0/2051628924 wait complete. 2024-03-18T15:34:00.654 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T15:33:28.963186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-18T15:33:40.533646+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-18T15:33:41.323519+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T15:34:00.654 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T15:34:00.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.765+0000 7fcacd9c9640 1 Processor -- start 2024-03-18T15:34:00.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.765+0000 7fcacd9c9640 1 -- start start 2024-03-18T15:34:00.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.769+0000 7fcacd9c9640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcac806d2d0 0x7fcac80b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:00.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.769+0000 7fcacd9c9640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcac813f260 0x7fcac813f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:00.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.769+0000 7fcacd9c9640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcac80b0ea0 con 0x7fcac81453a0 2024-03-18T15:34:00.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.769+0000 7fcacd9c9640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fcac80728c0 con 0x7fcac806d2d0 2024-03-18T15:34:00.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.769+0000 7fcacd9c9640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fcac806d6b0 con 0x7fcac813f260 2024-03-18T15:34:00.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.769+0000 7fcac6ffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcac806d2d0 0x7fcac80b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:00.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.769+0000 7fcac77fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fcac81453a0 0x7fcac8145810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60034/0 (socket says 172.21.15.5:60034) 2024-03-18T15:34:00.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.769+0000 7fcac77fe640 1 -- 172.21.15.5:0/632806043 learned_addr learned my addr 172.21.15.5:0/632806043 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:00.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.769+0000 7fcac67fc640 1 --2- 172.21.15.5:0/632806043 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcac813f260 0x7fcac813f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:00.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.769+0000 7fcac6ffd640 1 -- 172.21.15.5:0/632806043 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcac813f260 msgr2=0x7fcac813f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:00.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.769+0000 7fcac6ffd640 1 --2- 172.21.15.5:0/632806043 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcac813f260 0x7fcac813f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:00.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.769+0000 7fcac6ffd640 1 -- 172.21.15.5:0/632806043 >> v1:172.21.15.5:6789/0 conn(0x7fcac81453a0 legacy=0x7fcac8145810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:00.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.769+0000 7fcac6ffd640 1 -- 172.21.15.5:0/632806043 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcac80b0bc0 con 0x7fcac806d2d0 2024-03-18T15:34:00.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.769+0000 7fcac6ffd640 1 --2- 172.21.15.5:0/632806043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcac806d2d0 0x7fcac80b23a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcac0009d90 tx=0x7fcac001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfc808c0ec50b069 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:00.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.769+0000 7fcac5ffb640 1 -- 172.21.15.5:0/632806043 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac00b2070 con 0x7fcac806d2d0 2024-03-18T15:34:00.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.769+0000 7fcac5ffb640 1 -- 172.21.15.5:0/632806043 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcac00a58b0 con 0x7fcac806d2d0 2024-03-18T15:34:00.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.769+0000 7fcac5ffb640 1 -- 172.21.15.5:0/632806043 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac00a5c80 con 0x7fcac806d2d0 2024-03-18T15:34:00.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.769+0000 7fcacd9c9640 1 -- 172.21.15.5:0/632806043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcac806d2d0 msgr2=0x7fcac80b23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:00.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.769+0000 7fcacd9c9640 1 --2- 172.21.15.5:0/632806043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcac806d2d0 0x7fcac80b23a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcac0009d90 tx=0x7fcac001c8e0 comp rx=0 tx=0).stop 2024-03-18T15:34:00.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.769+0000 7fcacd9c9640 1 -- 172.21.15.5:0/632806043 shutdown_connections 2024-03-18T15:34:00.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.769+0000 7fcacd9c9640 1 --2- 172.21.15.5:0/632806043 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcac813f260 0x7fcac813f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:00.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.769+0000 7fcacd9c9640 1 --2- 172.21.15.5:0/632806043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcac806d2d0 0x7fcac80b23a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:00.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.769+0000 7fcacd9c9640 1 -- 172.21.15.5:0/632806043 >> 172.21.15.5:0/632806043 conn(0x7fcac80b5670 msgr2=0x7fcac80b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:00.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.769+0000 7fcacd9c9640 1 -- 172.21.15.5:0/632806043 shutdown_connections 2024-03-18T15:34:00.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.769+0000 7fcacd9c9640 1 -- 172.21.15.5:0/632806043 wait complete. 2024-03-18T15:34:00.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.773+0000 7fcacd9c9640 1 Processor -- start 2024-03-18T15:34:00.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.773+0000 7fcacd9c9640 1 -- start start 2024-03-18T15:34:00.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.773+0000 7fcacd9c9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcac806d2d0 0x7fcac8136b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:00.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.773+0000 7fcacd9c9640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcac813f260 0x7fcac80af250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:00.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.773+0000 7fcacd9c9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcac81453a0 0x7fcac81e36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:00.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.773+0000 7fcacd9c9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcac8072ed0 con 0x7fcac806d2d0 2024-03-18T15:34:00.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.773+0000 7fcacd9c9640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fcac8072d50 con 0x7fcac81453a0 2024-03-18T15:34:00.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.773+0000 7fcacd9c9640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fcac80b5140 con 0x7fcac813f260 2024-03-18T15:34:00.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.773+0000 7fcac6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcac806d2d0 0x7fcac8136b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:00.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.773+0000 7fcac6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcac806d2d0 0x7fcac8136b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49900/0 (socket says 172.21.15.5:49900) 2024-03-18T15:34:00.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.773+0000 7fcac6ffd640 1 -- 172.21.15.5:0/970503328 learned_addr learned my addr 172.21.15.5:0/970503328 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:00.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.773+0000 7fcac77fe640 1 --2- 172.21.15.5:0/970503328 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcac81453a0 0x7fcac81e36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:00.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.773+0000 7fcac67fc640 1 --2- 172.21.15.5:0/970503328 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcac813f260 0x7fcac80af250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:00.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.773+0000 7fcac6ffd640 1 -- 172.21.15.5:0/970503328 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcac813f260 msgr2=0x7fcac80af250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:00.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.773+0000 7fcac6ffd640 1 --2- 172.21.15.5:0/970503328 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcac813f260 0x7fcac80af250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:00.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.773+0000 7fcac6ffd640 1 -- 172.21.15.5:0/970503328 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcac81453a0 msgr2=0x7fcac81e36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:00.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.773+0000 7fcac6ffd640 1 --2- 172.21.15.5:0/970503328 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcac81453a0 0x7fcac81e36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:00.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.773+0000 7fcac6ffd640 1 -- 172.21.15.5:0/970503328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcac80b83f0 con 0x7fcac806d2d0 2024-03-18T15:34:00.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.773+0000 7fcac77fe640 1 --2- 172.21.15.5:0/970503328 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcac81453a0 0x7fcac81e36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:34:00.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.773+0000 7fcac67fc640 1 --2- 172.21.15.5:0/970503328 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcac813f260 0x7fcac80af250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:34:00.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.773+0000 7fcac6ffd640 1 --2- 172.21.15.5:0/970503328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcac806d2d0 0x7fcac8136b60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcac00b0040 tx=0x7fcac0019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:00.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.773+0000 7fcabffff640 1 -- 172.21.15.5:0/970503328 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac00b2050 con 0x7fcac806d2d0 2024-03-18T15:34:00.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.773+0000 7fcabffff640 1 -- 172.21.15.5:0/970503328 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcac00be070 con 0x7fcac806d2d0 2024-03-18T15:34:00.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.773+0000 7fcacd9c9640 1 -- 172.21.15.5:0/970503328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcac80af930 con 0x7fcac806d2d0 2024-03-18T15:34:00.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.773+0000 7fcabffff640 1 -- 172.21.15.5:0/970503328 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac00ada90 con 0x7fcac806d2d0 2024-03-18T15:34:00.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.773+0000 7fcacd9c9640 1 -- 172.21.15.5:0/970503328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcac81373c0 con 0x7fcac806d2d0 2024-03-18T15:34:00.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.773+0000 7fcacd9c9640 1 -- 172.21.15.5:0/970503328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcac804f550 con 0x7fcac806d2d0 2024-03-18T15:34:00.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.773+0000 7fcabffff640 1 -- 172.21.15.5:0/970503328 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fcac00adc30 con 0x7fcac806d2d0 2024-03-18T15:34:00.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.777+0000 7fcabffff640 1 --2- 172.21.15.5:0/970503328 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fcaa8067660 0x7fcaa8069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:00.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.777+0000 7fcabffff640 1 -- 172.21.15.5:0/970503328 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fcac0120950 con 0x7fcac806d2d0 2024-03-18T15:34:00.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.777+0000 7fcac67fc640 1 --2- 172.21.15.5:0/970503328 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fcaa8067660 0x7fcaa8069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:00.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.777+0000 7fcac67fc640 1 --2- 172.21.15.5:0/970503328 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fcaa8067660 0x7fcaa8069b20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcac80b2e00 tx=0x7fcab0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:00.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.781+0000 7fcabffff640 1 -- 172.21.15.5:0/970503328 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fcac0016560 con 0x7fcac806d2d0 2024-03-18T15:34:00.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.933+0000 7fcacd9c9640 1 -- 172.21.15.5:0/970503328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcac813bf60 con 0x7fcac806d2d0 2024-03-18T15:34:00.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.933+0000 7fcabffff640 1 -- 172.21.15.5:0/970503328 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7fcac00f16a0 con 0x7fcac806d2d0 2024-03-18T15:34:00.936 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-03-18T15:34:00.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.937+0000 7fcacd9c9640 1 -- 172.21.15.5:0/970503328 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fcaa8067660 msgr2=0x7fcaa8069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:00.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.937+0000 7fcacd9c9640 1 --2- 172.21.15.5:0/970503328 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fcaa8067660 0x7fcaa8069b20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcac80b2e00 tx=0x7fcab0008040 comp rx=0 tx=0).stop 2024-03-18T15:34:00.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.937+0000 7fcacd9c9640 1 -- 172.21.15.5:0/970503328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcac806d2d0 msgr2=0x7fcac8136b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:00.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.937+0000 7fcacd9c9640 1 --2- 172.21.15.5:0/970503328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcac806d2d0 0x7fcac8136b60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcac00b0040 tx=0x7fcac0019680 comp rx=0 tx=0).stop 2024-03-18T15:34:00.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.937+0000 7fcacd9c9640 1 -- 172.21.15.5:0/970503328 shutdown_connections 2024-03-18T15:34:00.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.937+0000 7fcacd9c9640 1 --2- 172.21.15.5:0/970503328 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fcaa8067660 0x7fcaa8069b20 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:00.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.937+0000 7fcacd9c9640 1 --2- 172.21.15.5:0/970503328 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcac81453a0 0x7fcac81e36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:00.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.937+0000 7fcacd9c9640 1 --2- 172.21.15.5:0/970503328 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcac813f260 0x7fcac80af250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:00.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.937+0000 7fcacd9c9640 1 --2- 172.21.15.5:0/970503328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcac806d2d0 0x7fcac8136b60 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:00.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.937+0000 7fcacd9c9640 1 -- 172.21.15.5:0/970503328 >> 172.21.15.5:0/970503328 conn(0x7fcac80b5670 msgr2=0x7fcac813b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:00.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.937+0000 7fcacd9c9640 1 -- 172.21.15.5:0/970503328 shutdown_connections 2024-03-18T15:34:00.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:00.937+0000 7fcacd9c9640 1 -- 172.21.15.5:0/970503328 wait complete. 2024-03-18T15:34:00.953 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-18T15:34:01.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.081+0000 7fd6c5e8f640 1 Processor -- start 2024-03-18T15:34:01.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.081+0000 7fd6c5e8f640 1 -- start start 2024-03-18T15:34:01.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.081+0000 7fd6c5e8f640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd6c0072b40 0x7fd6c006d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:01.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.081+0000 7fd6c5e8f640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd6c00b26a0 0x7fd6c00b2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:01.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.081+0000 7fd6c5e8f640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6c00b3070 con 0x7fd6c00af790 2024-03-18T15:34:01.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.081+0000 7fd6c5e8f640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fd6c006d5b0 con 0x7fd6c0072b40 2024-03-18T15:34:01.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.081+0000 7fd6c5e8f640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fd6c00afb70 con 0x7fd6c00b26a0 2024-03-18T15:34:01.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.081+0000 7fd6beffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd6c0072b40 0x7fd6c006d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:01.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.081+0000 7fd6beffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd6c0072b40 0x7fd6c006d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49904/0 (socket says 172.21.15.5:49904) 2024-03-18T15:34:01.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.081+0000 7fd6bffff640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd6c00b26a0 0x7fd6c00b2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:01.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.081+0000 7fd6beffd640 1 -- 172.21.15.5:0/624039522 learned_addr learned my addr 172.21.15.5:0/624039522 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:01.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.081+0000 7fd6beffd640 1 -- 172.21.15.5:0/624039522 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd6c00b26a0 msgr2=0x7fd6c00b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:01.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.081+0000 7fd6beffd640 1 --2- 172.21.15.5:0/624039522 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd6c00b26a0 0x7fd6c00b2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:01.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.081+0000 7fd6beffd640 1 -- 172.21.15.5:0/624039522 >> v1:172.21.15.80:6789/0 conn(0x7fd6c00af790 legacy=0x7fd6c0147780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:01.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.081+0000 7fd6beffd640 1 -- 172.21.15.5:0/624039522 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6c00b4a20 con 0x7fd6c0072b40 2024-03-18T15:34:01.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.085+0000 7fd6beffd640 1 --2- 172.21.15.5:0/624039522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6c0072b40 0x7fd6c006d070 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd6b4009870 tx=0x7fd6b401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=add6265ad58a66cf server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:01.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.085+0000 7fd6be7fc640 1 -- 172.21.15.5:0/624039522 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b40b0070 con 0x7fd6c0072b40 2024-03-18T15:34:01.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.085+0000 7fd6be7fc640 1 -- 172.21.15.5:0/624039522 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6b40ab600 con 0x7fd6c0072b40 2024-03-18T15:34:01.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.085+0000 7fd6be7fc640 1 -- 172.21.15.5:0/624039522 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b40ab9d0 con 0x7fd6c0072b40 2024-03-18T15:34:01.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.085+0000 7fd6c5e8f640 1 -- 172.21.15.5:0/624039522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6c0072b40 msgr2=0x7fd6c006d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:01.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.085+0000 7fd6c5e8f640 1 --2- 172.21.15.5:0/624039522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6c0072b40 0x7fd6c006d070 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd6b4009870 tx=0x7fd6b401c4d0 comp rx=0 tx=0).stop 2024-03-18T15:34:01.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.085+0000 7fd6c5e8f640 1 -- 172.21.15.5:0/624039522 shutdown_connections 2024-03-18T15:34:01.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.085+0000 7fd6c5e8f640 1 --2- 172.21.15.5:0/624039522 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd6c00b26a0 0x7fd6c00b2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:01.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.085+0000 7fd6c5e8f640 1 --2- 172.21.15.5:0/624039522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6c0072b40 0x7fd6c006d070 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:01.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.085+0000 7fd6c5e8f640 1 -- 172.21.15.5:0/624039522 >> 172.21.15.5:0/624039522 conn(0x7fd6c00b5670 msgr2=0x7fd6c00b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:01.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.085+0000 7fd6c5e8f640 1 -- 172.21.15.5:0/624039522 shutdown_connections 2024-03-18T15:34:01.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.085+0000 7fd6c5e8f640 1 -- 172.21.15.5:0/624039522 wait complete. 2024-03-18T15:34:01.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.085+0000 7fd6c5e8f640 1 Processor -- start 2024-03-18T15:34:01.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.085+0000 7fd6c5e8f640 1 -- start start 2024-03-18T15:34:01.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.085+0000 7fd6c5e8f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6c0072b40 0x7fd6c0144600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:01.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.085+0000 7fd6c5e8f640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd6c00af790 0x7fd6c0145ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:01.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.085+0000 7fd6c5e8f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd6c00b26a0 0x7fd6c01e30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:01.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.085+0000 7fd6c5e8f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c01516a0 con 0x7fd6c0072b40 2024-03-18T15:34:01.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.085+0000 7fd6c5e8f640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c0151520 con 0x7fd6c00b26a0 2024-03-18T15:34:01.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.085+0000 7fd6c5e8f640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fd6c0151820 con 0x7fd6c00af790 2024-03-18T15:34:01.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.085+0000 7fd6bf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6c0072b40 0x7fd6c0144600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:01.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.085+0000 7fd6bf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6c0072b40 0x7fd6c0144600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49912/0 (socket says 172.21.15.5:49912) 2024-03-18T15:34:01.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.085+0000 7fd6bf7fe640 1 -- 172.21.15.5:0/3375917068 learned_addr learned my addr 172.21.15.5:0/3375917068 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:01.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.085+0000 7fd6bffff640 1 --2- 172.21.15.5:0/3375917068 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd6c00b26a0 0x7fd6c01e30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:01.090 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.085+0000 7fd6beffd640 1 --2- 172.21.15.5:0/3375917068 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd6c00af790 0x7fd6c0145ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:01.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.085+0000 7fd6bf7fe640 1 -- 172.21.15.5:0/3375917068 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd6c00af790 msgr2=0x7fd6c0145ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:01.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.085+0000 7fd6bf7fe640 1 --2- 172.21.15.5:0/3375917068 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd6c00af790 0x7fd6c0145ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:01.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.085+0000 7fd6bf7fe640 1 -- 172.21.15.5:0/3375917068 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd6c00b26a0 msgr2=0x7fd6c01e30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:01.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.085+0000 7fd6bf7fe640 1 --2- 172.21.15.5:0/3375917068 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd6c00b26a0 0x7fd6c01e30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:01.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.085+0000 7fd6bf7fe640 1 -- 172.21.15.5:0/3375917068 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6c0146420 con 0x7fd6c0072b40 2024-03-18T15:34:01.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.085+0000 7fd6bf7fe640 1 --2- 172.21.15.5:0/3375917068 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6c0072b40 0x7fd6c0144600 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd6b000c420 tx=0x7fd6b000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:01.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.085+0000 7fd6a7fff640 1 -- 172.21.15.5:0/3375917068 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b00052d0 con 0x7fd6c0072b40 2024-03-18T15:34:01.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.085+0000 7fd6a7fff640 1 -- 172.21.15.5:0/3375917068 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6b0005470 con 0x7fd6c0072b40 2024-03-18T15:34:01.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.085+0000 7fd6c5e8f640 1 -- 172.21.15.5:0/3375917068 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6c01dfcd0 con 0x7fd6c0072b40 2024-03-18T15:34:01.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.085+0000 7fd6a7fff640 1 -- 172.21.15.5:0/3375917068 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b0007260 con 0x7fd6c0072b40 2024-03-18T15:34:01.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.085+0000 7fd6c5e8f640 1 -- 172.21.15.5:0/3375917068 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6c006f7d0 con 0x7fd6c0072b40 2024-03-18T15:34:01.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.089+0000 7fd6c5e8f640 1 -- 172.21.15.5:0/3375917068 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6c01e02c0 con 0x7fd6c0072b40 2024-03-18T15:34:01.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.089+0000 7fd6a7fff640 1 -- 172.21.15.5:0/3375917068 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fd6b0007400 con 0x7fd6c0072b40 2024-03-18T15:34:01.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.089+0000 7fd6a7fff640 1 --2- 172.21.15.5:0/3375917068 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd6ac067660 0x7fd6ac069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:01.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.089+0000 7fd6a7fff640 1 -- 172.21.15.5:0/3375917068 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd6b0084cf0 con 0x7fd6c0072b40 2024-03-18T15:34:01.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.089+0000 7fd6beffd640 1 --2- 172.21.15.5:0/3375917068 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd6ac067660 0x7fd6ac069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:01.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.089+0000 7fd6beffd640 1 --2- 172.21.15.5:0/3375917068 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd6ac067660 0x7fd6ac069b20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd6b40ad000 tx=0x7fd6b4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:01.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.093+0000 7fd6a7fff640 1 -- 172.21.15.5:0/3375917068 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fd6b0089070 con 0x7fd6c0072b40 2024-03-18T15:34:01.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.249+0000 7fd6c5e8f640 1 -- 172.21.15.5:0/3375917068 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd6c004f550 con 0x7fd6c0072b40 2024-03-18T15:34:01.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.249+0000 7fd6a7fff640 1 -- 172.21.15.5:0/3375917068 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7fd6b0055a00 con 0x7fd6c0072b40 2024-03-18T15:34:01.251 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-03-18T15:34:01.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.249+0000 7fd6c5e8f640 1 -- 172.21.15.5:0/3375917068 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd6ac067660 msgr2=0x7fd6ac069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:01.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.249+0000 7fd6c5e8f640 1 --2- 172.21.15.5:0/3375917068 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd6ac067660 0x7fd6ac069b20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd6b40ad000 tx=0x7fd6b4008040 comp rx=0 tx=0).stop 2024-03-18T15:34:01.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.249+0000 7fd6c5e8f640 1 -- 172.21.15.5:0/3375917068 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6c0072b40 msgr2=0x7fd6c0144600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:01.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.249+0000 7fd6c5e8f640 1 --2- 172.21.15.5:0/3375917068 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6c0072b40 0x7fd6c0144600 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd6b000c420 tx=0x7fd6b000c8e0 comp rx=0 tx=0).stop 2024-03-18T15:34:01.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.249+0000 7fd6c5e8f640 1 -- 172.21.15.5:0/3375917068 shutdown_connections 2024-03-18T15:34:01.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.249+0000 7fd6c5e8f640 1 --2- 172.21.15.5:0/3375917068 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd6ac067660 0x7fd6ac069b20 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:01.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.249+0000 7fd6c5e8f640 1 --2- 172.21.15.5:0/3375917068 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd6c00b26a0 0x7fd6c01e30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:01.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.249+0000 7fd6c5e8f640 1 --2- 172.21.15.5:0/3375917068 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd6c00af790 0x7fd6c0145ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:01.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.249+0000 7fd6c5e8f640 1 --2- 172.21.15.5:0/3375917068 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6c0072b40 0x7fd6c0144600 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:01.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.249+0000 7fd6c5e8f640 1 -- 172.21.15.5:0/3375917068 >> 172.21.15.5:0/3375917068 conn(0x7fd6c00b5670 msgr2=0x7fd6c00adcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:01.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.253+0000 7fd6c5e8f640 1 -- 172.21.15.5:0/3375917068 shutdown_connections 2024-03-18T15:34:01.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.253+0000 7fd6c5e8f640 1 -- 172.21.15.5:0/3375917068 wait complete. 2024-03-18T15:34:01.268 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-18T15:34:01.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.393+0000 7fe1bedb4640 1 Processor -- start 2024-03-18T15:34:01.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.393+0000 7fe1bedb4640 1 -- start start 2024-03-18T15:34:01.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.393+0000 7fe1bedb4640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe1b814b030 0x7fe1b8140ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:01.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.393+0000 7fe1bedb4640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1b80b9d50 con 0x7fe1b80b5710 2024-03-18T15:34:01.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.393+0000 7fe1bedb4640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1b80b4c30 con 0x7fe1b8141430 2024-03-18T15:34:01.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.393+0000 7fe1bedb4640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fe1b80b4320 con 0x7fe1b814b030 2024-03-18T15:34:01.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.393+0000 7fe1b7fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe1b814b030 0x7fe1b8140ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:01.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.393+0000 7fe1b7fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe1b814b030 0x7fe1b8140ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49926/0 (socket says 172.21.15.5:49926) 2024-03-18T15:34:01.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.393+0000 7fe1b7fff640 1 -- 172.21.15.5:0/2057790508 learned_addr learned my addr 172.21.15.5:0/2057790508 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:01.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.393+0000 7fe1bcb50640 1 --1- 172.21.15.5:0/2057790508 >> v1:172.21.15.5:6789/0 conn(0x7fe1b80b5710 0x7fe1b814a920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60036/0 (socket says 172.21.15.5:60036) 2024-03-18T15:34:01.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.393+0000 7fe1b7fff640 1 -- 172.21.15.5:0/2057790508 >> v1:172.21.15.80:6789/0 conn(0x7fe1b8141430 legacy=0x7fe1b8151cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:01.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.393+0000 7fe1b7fff640 1 -- 172.21.15.5:0/2057790508 >> v1:172.21.15.5:6789/0 conn(0x7fe1b80b5710 legacy=0x7fe1b814a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:01.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.393+0000 7fe1b7fff640 1 -- 172.21.15.5:0/2057790508 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1b80b9210 con 0x7fe1b814b030 2024-03-18T15:34:01.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.393+0000 7fe1b77fe640 1 -- 172.21.15.5:0/2057790508 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4096985533 0 0) 0x7fe1b80b9d50 con 0x7fe1b80b5710 2024-03-18T15:34:01.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.397+0000 7fe1b7fff640 1 --2- 172.21.15.5:0/2057790508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1b814b030 0x7fe1b8140ef0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4009870 tx=0x7fe1a401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ff29a6261304b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:01.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.397+0000 7fe1b77fe640 1 -- 172.21.15.5:0/2057790508 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a40b0070 con 0x7fe1b814b030 2024-03-18T15:34:01.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.397+0000 7fe1b77fe640 1 -- 172.21.15.5:0/2057790508 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1a40ab600 con 0x7fe1b814b030 2024-03-18T15:34:01.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.397+0000 7fe1b77fe640 1 -- 172.21.15.5:0/2057790508 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1a40ab9d0 con 0x7fe1b814b030 2024-03-18T15:34:01.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.397+0000 7fe1bedb4640 1 -- 172.21.15.5:0/2057790508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1b814b030 msgr2=0x7fe1b8140ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:01.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.397+0000 7fe1bedb4640 1 --2- 172.21.15.5:0/2057790508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1b814b030 0x7fe1b8140ef0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe1a4009870 tx=0x7fe1a401c440 comp rx=0 tx=0).stop 2024-03-18T15:34:01.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.397+0000 7fe1bedb4640 1 -- 172.21.15.5:0/2057790508 shutdown_connections 2024-03-18T15:34:01.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.397+0000 7fe1bedb4640 1 --2- 172.21.15.5:0/2057790508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1b814b030 0x7fe1b8140ef0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:01.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.397+0000 7fe1bedb4640 1 -- 172.21.15.5:0/2057790508 >> 172.21.15.5:0/2057790508 conn(0x7fe1b80673c0 msgr2=0x7fe1b80677d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:01.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.397+0000 7fe1bedb4640 1 -- 172.21.15.5:0/2057790508 shutdown_connections 2024-03-18T15:34:01.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.397+0000 7fe1bedb4640 1 -- 172.21.15.5:0/2057790508 wait complete. 2024-03-18T15:34:01.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.397+0000 7fe1bedb4640 1 Processor -- start 2024-03-18T15:34:01.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.397+0000 7fe1bedb4640 1 -- start start 2024-03-18T15:34:01.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.397+0000 7fe1bedb4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1b80b5710 0x7fe1b813a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:01.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.397+0000 7fe1bedb4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1b8141430 0x7fe1b8132c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:01.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.397+0000 7fe1bedb4640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1b814b030 0x7fe1b81e3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:01.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.397+0000 7fe1bedb4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b8155bf0 con 0x7fe1b80b5710 2024-03-18T15:34:01.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.397+0000 7fe1bedb4640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe1b80b6940 con 0x7fe1b8141430 2024-03-18T15:34:01.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.397+0000 7fe1bedb4640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fe1b8155d70 con 0x7fe1b814b030 2024-03-18T15:34:01.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.397+0000 7fe1bcb50640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1b80b5710 0x7fe1b813a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:01.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.397+0000 7fe1bcb50640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1b80b5710 0x7fe1b813a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49932/0 (socket says 172.21.15.5:49932) 2024-03-18T15:34:01.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.397+0000 7fe1bcb50640 1 -- 172.21.15.5:0/3163546794 learned_addr learned my addr 172.21.15.5:0/3163546794 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:01.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.397+0000 7fe1bd351640 1 --2- 172.21.15.5:0/3163546794 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1b814b030 0x7fe1b81e3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:01.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.397+0000 7fe1b7fff640 1 --2- 172.21.15.5:0/3163546794 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1b8141430 0x7fe1b8132c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:01.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.397+0000 7fe1bcb50640 1 -- 172.21.15.5:0/3163546794 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1b814b030 msgr2=0x7fe1b81e3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:01.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.397+0000 7fe1bcb50640 1 --2- 172.21.15.5:0/3163546794 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1b814b030 0x7fe1b81e3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:01.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.397+0000 7fe1bcb50640 1 -- 172.21.15.5:0/3163546794 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1b8141430 msgr2=0x7fe1b8132c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:01.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.397+0000 7fe1bcb50640 1 --2- 172.21.15.5:0/3163546794 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1b8141430 0x7fe1b8132c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:01.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.397+0000 7fe1bcb50640 1 -- 172.21.15.5:0/3163546794 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1b81344b0 con 0x7fe1b80b5710 2024-03-18T15:34:01.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.401+0000 7fe1bcb50640 1 --2- 172.21.15.5:0/3163546794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1b80b5710 0x7fe1b813a290 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac00c440 tx=0x7fe1ac00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:01.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.401+0000 7fe1b57fa640 1 -- 172.21.15.5:0/3163546794 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac018070 con 0x7fe1b80b5710 2024-03-18T15:34:01.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.401+0000 7fe1b57fa640 1 -- 172.21.15.5:0/3163546794 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1ac0074e0 con 0x7fe1b80b5710 2024-03-18T15:34:01.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.401+0000 7fe1bedb4640 1 -- 172.21.15.5:0/3163546794 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1b80bbfc0 con 0x7fe1b80b5710 2024-03-18T15:34:01.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.401+0000 7fe1b57fa640 1 -- 172.21.15.5:0/3163546794 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac004a10 con 0x7fe1b80b5710 2024-03-18T15:34:01.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.401+0000 7fe1bedb4640 1 -- 172.21.15.5:0/3163546794 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1b8134e70 con 0x7fe1b80b5710 2024-03-18T15:34:01.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.401+0000 7fe1bedb4640 1 -- 172.21.15.5:0/3163546794 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1b804f550 con 0x7fe1b80b5710 2024-03-18T15:34:01.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.401+0000 7fe1b57fa640 1 -- 172.21.15.5:0/3163546794 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fe1ac01b070 con 0x7fe1b80b5710 2024-03-18T15:34:01.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.401+0000 7fe1b57fa640 1 --2- 172.21.15.5:0/3163546794 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe19c067660 0x7fe19c069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:01.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.401+0000 7fe1b7fff640 1 --2- 172.21.15.5:0/3163546794 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe19c067660 0x7fe19c069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:01.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.401+0000 7fe1b57fa640 1 -- 172.21.15.5:0/3163546794 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe1ac084a60 con 0x7fe1b80b5710 2024-03-18T15:34:01.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.401+0000 7fe1b7fff640 1 --2- 172.21.15.5:0/3163546794 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe19c067660 0x7fe19c069b20 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe1a40ad000 tx=0x7fe1a40a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:01.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.405+0000 7fe1b57fa640 1 -- 172.21.15.5:0/3163546794 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fe1ac055750 con 0x7fe1b80b5710 2024-03-18T15:34:01.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.561+0000 7fe1bedb4640 1 -- 172.21.15.5:0/3163546794 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe1b8144d50 con 0x7fe1b80b5710 2024-03-18T15:34:01.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.565+0000 7fe1b57fa640 1 -- 172.21.15.5:0/3163546794 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7fe1ac055050 con 0x7fe1b80b5710 2024-03-18T15:34:01.566 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-03-18T15:34:01.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.565+0000 7fe1bedb4640 1 -- 172.21.15.5:0/3163546794 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe19c067660 msgr2=0x7fe19c069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:01.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.565+0000 7fe1bedb4640 1 --2- 172.21.15.5:0/3163546794 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe19c067660 0x7fe19c069b20 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe1a40ad000 tx=0x7fe1a40a6000 comp rx=0 tx=0).stop 2024-03-18T15:34:01.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.565+0000 7fe1bedb4640 1 -- 172.21.15.5:0/3163546794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1b80b5710 msgr2=0x7fe1b813a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:01.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.565+0000 7fe1bedb4640 1 --2- 172.21.15.5:0/3163546794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1b80b5710 0x7fe1b813a290 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac00c440 tx=0x7fe1ac00c900 comp rx=0 tx=0).stop 2024-03-18T15:34:01.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.565+0000 7fe1bedb4640 1 -- 172.21.15.5:0/3163546794 shutdown_connections 2024-03-18T15:34:01.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.565+0000 7fe1bedb4640 1 --2- 172.21.15.5:0/3163546794 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe19c067660 0x7fe19c069b20 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:01.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.565+0000 7fe1bedb4640 1 --2- 172.21.15.5:0/3163546794 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1b814b030 0x7fe1b81e3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:01.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.565+0000 7fe1bedb4640 1 --2- 172.21.15.5:0/3163546794 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1b8141430 0x7fe1b8132c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:01.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.565+0000 7fe1bedb4640 1 --2- 172.21.15.5:0/3163546794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1b80b5710 0x7fe1b813a290 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:01.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.565+0000 7fe1bedb4640 1 -- 172.21.15.5:0/3163546794 >> 172.21.15.5:0/3163546794 conn(0x7fe1b80673c0 msgr2=0x7fe1b80b7de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:01.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.565+0000 7fe1bedb4640 1 -- 172.21.15.5:0/3163546794 shutdown_connections 2024-03-18T15:34:01.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.565+0000 7fe1bedb4640 1 -- 172.21.15.5:0/3163546794 wait complete. 2024-03-18T15:34:01.584 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:34:01.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.705+0000 7f20bd394640 1 Processor -- start 2024-03-18T15:34:01.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.705+0000 7f20bd394640 1 -- start start 2024-03-18T15:34:01.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.705+0000 7f20bd394640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f20b8072890 0x7f20b80b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:01.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.705+0000 7f20bd394640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20b80af100 con 0x7f20b813f7f0 2024-03-18T15:34:01.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.705+0000 7f20bd394640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20b8072c70 con 0x7f20b81488b0 2024-03-18T15:34:01.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.705+0000 7f20bd394640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f20b80af370 con 0x7f20b8072890 2024-03-18T15:34:01.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.705+0000 7f20b6ffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f20b8072890 0x7f20b80b4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:01.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.705+0000 7f20b77fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f20b813f7f0 0x7f20b814fbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60046/0 (socket says 172.21.15.5:60046) 2024-03-18T15:34:01.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.705+0000 7f20b77fe640 1 -- 172.21.15.5:0/681994293 learned_addr learned my addr 172.21.15.5:0/681994293 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:01.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.705+0000 7f20b6ffd640 1 -- 172.21.15.5:0/681994293 >> v1:172.21.15.80:6789/0 conn(0x7f20b81488b0 legacy=0x7f20b813f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:01.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.705+0000 7f20b5ffb640 1 -- 172.21.15.5:0/681994293 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 545187915 0 0) 0x7f20b80af100 con 0x7f20b813f7f0 2024-03-18T15:34:01.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.705+0000 7f20b6ffd640 1 -- 172.21.15.5:0/681994293 >> v1:172.21.15.5:6789/0 conn(0x7f20b813f7f0 legacy=0x7f20b814fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:01.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.705+0000 7f20b6ffd640 1 -- 172.21.15.5:0/681994293 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b8148dd0 con 0x7f20b8072890 2024-03-18T15:34:01.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.709+0000 7f20b6ffd640 1 --2- 172.21.15.5:0/681994293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20b8072890 0x7f20b80b4da0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f20ac0097c0 tx=0x7f20ac01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2718d75703606f8d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:01.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.709+0000 7f20b5ffb640 1 -- 172.21.15.5:0/681994293 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ac0b0070 con 0x7f20b8072890 2024-03-18T15:34:01.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.709+0000 7f20b5ffb640 1 -- 172.21.15.5:0/681994293 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20ac0ab600 con 0x7f20b8072890 2024-03-18T15:34:01.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.709+0000 7f20b5ffb640 1 -- 172.21.15.5:0/681994293 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ac0ab9d0 con 0x7f20b8072890 2024-03-18T15:34:01.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.709+0000 7f20bd394640 1 -- 172.21.15.5:0/681994293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20b8072890 msgr2=0x7f20b80b4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:01.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.709+0000 7f20bd394640 1 --2- 172.21.15.5:0/681994293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20b8072890 0x7f20b80b4da0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f20ac0097c0 tx=0x7f20ac01c3e0 comp rx=0 tx=0).stop 2024-03-18T15:34:01.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.709+0000 7f20bd394640 1 -- 172.21.15.5:0/681994293 shutdown_connections 2024-03-18T15:34:01.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.709+0000 7f20bd394640 1 --2- 172.21.15.5:0/681994293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20b8072890 0x7f20b80b4da0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:01.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.709+0000 7f20bd394640 1 -- 172.21.15.5:0/681994293 >> 172.21.15.5:0/681994293 conn(0x7f20b806e230 msgr2=0x7f20b806e640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:01.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.709+0000 7f20bd394640 1 -- 172.21.15.5:0/681994293 shutdown_connections 2024-03-18T15:34:01.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.709+0000 7f20bd394640 1 -- 172.21.15.5:0/681994293 wait complete. 2024-03-18T15:34:01.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.709+0000 7f20bd394640 1 Processor -- start 2024-03-18T15:34:01.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.709+0000 7f20bd394640 1 -- start start 2024-03-18T15:34:01.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.713+0000 7f20bd394640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20b8072890 0x7f20b81e37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:01.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.713+0000 7f20bd394640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f20b813f7f0 0x7f20b80b9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:01.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.713+0000 7f20bd394640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f20b81488b0 0x7f20b809f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:01.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.713+0000 7f20bd394640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f20b8153ca0 con 0x7f20b8072890 2024-03-18T15:34:01.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.713+0000 7f20bd394640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f20b8153b20 con 0x7f20b813f7f0 2024-03-18T15:34:01.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.713+0000 7f20bd394640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f20b8153e20 con 0x7f20b81488b0 2024-03-18T15:34:01.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.713+0000 7f20b6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20b8072890 0x7f20b81e37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:01.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.713+0000 7f20b6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20b8072890 0x7f20b81e37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49942/0 (socket says 172.21.15.5:49942) 2024-03-18T15:34:01.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.713+0000 7f20b6ffd640 1 -- 172.21.15.5:0/4217243350 learned_addr learned my addr 172.21.15.5:0/4217243350 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:01.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.713+0000 7f20b77fe640 1 --2- 172.21.15.5:0/4217243350 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f20b81488b0 0x7f20b809f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:01.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.713+0000 7f20b67fc640 1 --2- 172.21.15.5:0/4217243350 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f20b813f7f0 0x7f20b80b9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:01.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.713+0000 7f20b6ffd640 1 -- 172.21.15.5:0/4217243350 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f20b81488b0 msgr2=0x7f20b809f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:01.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.713+0000 7f20b6ffd640 1 --2- 172.21.15.5:0/4217243350 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f20b81488b0 0x7f20b809f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:01.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.713+0000 7f20b6ffd640 1 -- 172.21.15.5:0/4217243350 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f20b813f7f0 msgr2=0x7f20b80b9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:01.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.713+0000 7f20b6ffd640 1 --2- 172.21.15.5:0/4217243350 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f20b813f7f0 0x7f20b80b9150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:01.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.713+0000 7f20b6ffd640 1 -- 172.21.15.5:0/4217243350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b8146950 con 0x7f20b8072890 2024-03-18T15:34:01.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.713+0000 7f20b77fe640 1 --2- 172.21.15.5:0/4217243350 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f20b81488b0 0x7f20b809f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:01.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.713+0000 7f20b67fc640 1 --2- 172.21.15.5:0/4217243350 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f20b813f7f0 0x7f20b80b9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:01.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.713+0000 7f20b6ffd640 1 --2- 172.21.15.5:0/4217243350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20b8072890 0x7f20b81e37d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f20ac0ae040 tx=0x7f20ac01ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:01.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.713+0000 7f209f7fe640 1 -- 172.21.15.5:0/4217243350 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ac0b0050 con 0x7f20b8072890 2024-03-18T15:34:01.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.713+0000 7f209f7fe640 1 -- 172.21.15.5:0/4217243350 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20ac0bc070 con 0x7f20b8072890 2024-03-18T15:34:01.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.713+0000 7f20bd394640 1 -- 172.21.15.5:0/4217243350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20b8146c30 con 0x7f20b8072890 2024-03-18T15:34:01.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.713+0000 7f209f7fe640 1 -- 172.21.15.5:0/4217243350 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ac0b5a70 con 0x7f20b8072890 2024-03-18T15:34:01.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.713+0000 7f20bd394640 1 -- 172.21.15.5:0/4217243350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20b80b9690 con 0x7f20b8072890 2024-03-18T15:34:01.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.713+0000 7f20bd394640 1 -- 172.21.15.5:0/4217243350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20b804f550 con 0x7f20b8072890 2024-03-18T15:34:01.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.713+0000 7f209f7fe640 1 -- 172.21.15.5:0/4217243350 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f20ac0ab480 con 0x7f20b8072890 2024-03-18T15:34:01.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.713+0000 7f209f7fe640 1 --2- 172.21.15.5:0/4217243350 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f20a4067660 0x7f20a4069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:01.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.713+0000 7f20b67fc640 1 --2- 172.21.15.5:0/4217243350 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f20a4067660 0x7f20a4069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:01.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.713+0000 7f209f7fe640 1 -- 172.21.15.5:0/4217243350 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f20ac120100 con 0x7f20b8072890 2024-03-18T15:34:01.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.717+0000 7f20b67fc640 1 --2- 172.21.15.5:0/4217243350 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f20a4067660 0x7f20a4069b20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f20a8007fd0 tx=0x7f20a8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:01.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.717+0000 7f209f7fe640 1 -- 172.21.15.5:0/4217243350 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f20ac016090 con 0x7f20b8072890 2024-03-18T15:34:01.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.877+0000 7f20bd394640 1 -- 172.21.15.5:0/4217243350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f20b80b5ef0 con 0x7f20b8072890 2024-03-18T15:34:01.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.881+0000 7f209f7fe640 1 -- 172.21.15.5:0/4217243350 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16449 (secure 0 0 0) 0x7f20ac0f0e10 con 0x7f20b8072890 2024-03-18T15:34:01.883 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:34:01.883 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":21,"fsid":"ef54fe03-5aa7-4eb2-9b1e-66c9cc345b94","created":"2024-03-18T15:33:18.086614+0000","modified":"2024-03-18T15:33:43.984050+0000","last_up_change":"2024-03-18T15:33:29.229911+0000","last_in_change":"2024-03-18T15:33:20.323840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:28.963186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T15:33:40.533646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T15:33:41.323519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df1c754b-af59-4d16-8919-fbb6386b3c48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6804","nonce":976264754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6808","nonce":976264754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6815","nonce":976264754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6811","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6813","nonce":976264754}]},"public_addr":"172.21.15.5:6804/976264754","cluster_addr":"172.21.15.5:6808/976264754","heartbeat_back_addr":"172.21.15.5:6815/976264754","heartbeat_front_addr":"172.21.15.5:6813/976264754","state":["exists","up"]},{"osd":1,"uuid":"b9a08b5e-8af4-4144-bdba-d69ed1ac627a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6801","nonce":3901761126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6803","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6805","nonce":3901761126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6812","nonce":3901761126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6807","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6809","nonce":3901761126}]},"public_addr":"172.21.15.5:6801/3901761126","cluster_addr":"172.21.15.5:6805/3901761126","heartbeat_back_addr":"172.21.15.5:6812/3901761126","heartbeat_front_addr":"172.21.15.5:6809/3901761126","state":["exists","up"]},{"osd":2,"uuid":"c3e8bb76-e382-439d-b380-62ef92f1255a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6817","nonce":502968448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6819","nonce":502968448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6823","nonce":502968448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6821","nonce":502968448}]},"public_addr":"172.21.15.5:6817/502968448","cluster_addr":"172.21.15.5:6819/502968448","heartbeat_back_addr":"172.21.15.5:6823/502968448","heartbeat_front_addr":"172.21.15.5:6821/502968448","state":["exists","up"]},{"osd":3,"uuid":"e8c94625-f2cd-4eba-b53d-2e7356e1c02a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6825","nonce":2425932735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6827","nonce":2425932735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6831","nonce":2425932735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6829","nonce":2425932735}]},"public_addr":"172.21.15.5:6825/2425932735","cluster_addr":"172.21.15.5:6827/2425932735","heartbeat_back_addr":"172.21.15.5:6831/2425932735","heartbeat_front_addr":"172.21.15.5:6829/2425932735","state":["exists","up"]},{"osd":4,"uuid":"0099dd52-6265-4d32-82e4-3ef16e73add0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6817","nonce":976514285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6819","nonce":976514285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6823","nonce":976514285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6821","nonce":976514285}]},"public_addr":"172.21.15.80:6817/976514285","cluster_addr":"172.21.15.80:6819/976514285","heartbeat_back_addr":"172.21.15.80:6823/976514285","heartbeat_front_addr":"172.21.15.80:6821/976514285","state":["exists","up"]},{"osd":5,"uuid":"ed10c84f-8c39-41cc-9b64-7c482a94a48e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6825","nonce":627153447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6827","nonce":627153447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6831","nonce":627153447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6829","nonce":627153447}]},"public_addr":"172.21.15.80:6825/627153447","cluster_addr":"172.21.15.80:6827/627153447","heartbeat_back_addr":"172.21.15.80:6831/627153447","heartbeat_front_addr":"172.21.15.80:6829/627153447","state":["exists","up"]},{"osd":6,"uuid":"c7a3538a-7048-4fba-8781-b440f03c25db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6803","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6805","nonce":488342527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6807","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6809","nonce":488342527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6815","nonce":488342527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6811","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6813","nonce":488342527}]},"public_addr":"172.21.15.80:6805/488342527","cluster_addr":"172.21.15.80:6809/488342527","heartbeat_back_addr":"172.21.15.80:6815/488342527","heartbeat_front_addr":"172.21.15.80:6813/488342527","state":["exists","up"]},{"osd":7,"uuid":"f9418104-3a97-4110-bc3a-1f8151d0f757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6801","nonce":982831666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6804","nonce":982831666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6812","nonce":982831666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6808","nonce":982831666}]},"public_addr":"172.21.15.80:6801/982831666","cluster_addr":"172.21.15.80:6804/982831666","heartbeat_back_addr":"172.21.15.80:6812/982831666","heartbeat_front_addr":"172.21.15.80:6808/982831666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.970828+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.858116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.987556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.864603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.756293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.591502+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:34:01.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.881+0000 7f20bd394640 1 -- 172.21.15.5:0/4217243350 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f20a4067660 msgr2=0x7f20a4069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:01.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.881+0000 7f20bd394640 1 --2- 172.21.15.5:0/4217243350 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f20a4067660 0x7f20a4069b20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f20a8007fd0 tx=0x7f20a8009040 comp rx=0 tx=0).stop 2024-03-18T15:34:01.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.881+0000 7f20bd394640 1 -- 172.21.15.5:0/4217243350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20b8072890 msgr2=0x7f20b81e37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:01.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.881+0000 7f20bd394640 1 --2- 172.21.15.5:0/4217243350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20b8072890 0x7f20b81e37d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f20ac0ae040 tx=0x7f20ac01ac20 comp rx=0 tx=0).stop 2024-03-18T15:34:01.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.881+0000 7f20bd394640 1 -- 172.21.15.5:0/4217243350 shutdown_connections 2024-03-18T15:34:01.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.881+0000 7f20bd394640 1 --2- 172.21.15.5:0/4217243350 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f20a4067660 0x7f20a4069b20 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:01.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.881+0000 7f20bd394640 1 --2- 172.21.15.5:0/4217243350 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f20b81488b0 0x7f20b809f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:01.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.881+0000 7f20bd394640 1 --2- 172.21.15.5:0/4217243350 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f20b813f7f0 0x7f20b80b9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:01.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.881+0000 7f20bd394640 1 --2- 172.21.15.5:0/4217243350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20b8072890 0x7f20b81e37d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:01.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.881+0000 7f20bd394640 1 -- 172.21.15.5:0/4217243350 >> 172.21.15.5:0/4217243350 conn(0x7f20b806e230 msgr2=0x7f20b8144ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:01.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.881+0000 7f20bd394640 1 -- 172.21.15.5:0/4217243350 shutdown_connections 2024-03-18T15:34:01.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:01.881+0000 7f20bd394640 1 -- 172.21.15.5:0/4217243350 wait complete. 2024-03-18T15:34:01.900 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T15:33:28.963186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-18T15:33:40.533646+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-18T15:33:41.323519+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T15:34:01.901 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T15:34:02.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.025+0000 7fd6f874a640 1 Processor -- start 2024-03-18T15:34:02.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.025+0000 7fd6f874a640 1 -- start start 2024-03-18T15:34:02.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.025+0000 7fd6f874a640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd6f00af790 0x7fd6f0072b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:02.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.025+0000 7fd6f874a640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd6f006d070 0x7fd6f0147780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:02.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.025+0000 7fd6f874a640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6f00afb70 con 0x7fd6f00b26a0 2024-03-18T15:34:02.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.025+0000 7fd6f874a640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fd6f00b4a20 con 0x7fd6f006d070 2024-03-18T15:34:02.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.025+0000 7fd6f874a640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fd6f00b4d00 con 0x7fd6f00af790 2024-03-18T15:34:02.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.025+0000 7fd6f5ce5640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd6f006d070 0x7fd6f0147780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:02.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.025+0000 7fd6f5ce5640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd6f006d070 0x7fd6f0147780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49944/0 (socket says 172.21.15.5:49944) 2024-03-18T15:34:02.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.025+0000 7fd6f5ce5640 1 -- 172.21.15.5:0/1526571630 learned_addr learned my addr 172.21.15.5:0/1526571630 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:02.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.029+0000 7fd6f64e6640 1 --2- 172.21.15.5:0/1526571630 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd6f00af790 0x7fd6f0072b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:02.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.029+0000 7fd6f5ce5640 1 -- 172.21.15.5:0/1526571630 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd6f00af790 msgr2=0x7fd6f0072b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:02.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.029+0000 7fd6f5ce5640 1 --2- 172.21.15.5:0/1526571630 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd6f00af790 0x7fd6f0072b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:02.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.029+0000 7fd6f5ce5640 1 -- 172.21.15.5:0/1526571630 >> v1:172.21.15.80:6789/0 conn(0x7fd6f00b26a0 legacy=0x7fd6f00b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:02.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.029+0000 7fd6f5ce5640 1 -- 172.21.15.5:0/1526571630 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f006d4b0 con 0x7fd6f006d070 2024-03-18T15:34:02.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.029+0000 7fd6f64e6640 1 --2- 172.21.15.5:0/1526571630 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd6f00af790 0x7fd6f0072b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:34:02.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.029+0000 7fd6f5ce5640 1 --2- 172.21.15.5:0/1526571630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6f006d070 0x7fd6f0147780 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0009870 tx=0x7fd6e001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57fdc2e5ca176957 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:02.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.029+0000 7fd6f54e4640 1 -- 172.21.15.5:0/1526571630 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e00b0070 con 0x7fd6f006d070 2024-03-18T15:34:02.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.029+0000 7fd6f54e4640 1 -- 172.21.15.5:0/1526571630 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6e00ab600 con 0x7fd6f006d070 2024-03-18T15:34:02.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.029+0000 7fd6f54e4640 1 -- 172.21.15.5:0/1526571630 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e00ab9d0 con 0x7fd6f006d070 2024-03-18T15:34:02.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.029+0000 7fd6f874a640 1 -- 172.21.15.5:0/1526571630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6f006d070 msgr2=0x7fd6f0147780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:02.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.029+0000 7fd6f874a640 1 --2- 172.21.15.5:0/1526571630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6f006d070 0x7fd6f0147780 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0009870 tx=0x7fd6e001c4d0 comp rx=0 tx=0).stop 2024-03-18T15:34:02.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.029+0000 7fd6f874a640 1 -- 172.21.15.5:0/1526571630 shutdown_connections 2024-03-18T15:34:02.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.029+0000 7fd6f874a640 1 --2- 172.21.15.5:0/1526571630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6f006d070 0x7fd6f0147780 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:02.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.029+0000 7fd6f874a640 1 --2- 172.21.15.5:0/1526571630 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd6f00af790 0x7fd6f0072b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:02.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.029+0000 7fd6f874a640 1 -- 172.21.15.5:0/1526571630 >> 172.21.15.5:0/1526571630 conn(0x7fd6f00b5670 msgr2=0x7fd6f00b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:02.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.029+0000 7fd6f874a640 1 -- 172.21.15.5:0/1526571630 shutdown_connections 2024-03-18T15:34:02.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.029+0000 7fd6f874a640 1 -- 172.21.15.5:0/1526571630 wait complete. 2024-03-18T15:34:02.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.033+0000 7fd6f874a640 1 Processor -- start 2024-03-18T15:34:02.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.033+0000 7fd6f874a640 1 -- start start 2024-03-18T15:34:02.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.033+0000 7fd6f874a640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd6f006d070 0x7fd6f0135c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:02.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.033+0000 7fd6f874a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6f00af790 0x7fd6f0144e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:02.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.033+0000 7fd6f874a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd6f00b26a0 0x7fd6f01dd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:02.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.033+0000 7fd6f874a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f01516a0 con 0x7fd6f00af790 2024-03-18T15:34:02.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.033+0000 7fd6f874a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f0151520 con 0x7fd6f00b26a0 2024-03-18T15:34:02.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.033+0000 7fd6f874a640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fd6f0151820 con 0x7fd6f006d070 2024-03-18T15:34:02.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.033+0000 7fd6f5ce5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6f00af790 0x7fd6f0144e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:02.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.033+0000 7fd6f5ce5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6f00af790 0x7fd6f0144e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49960/0 (socket says 172.21.15.5:49960) 2024-03-18T15:34:02.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.033+0000 7fd6f5ce5640 1 -- 172.21.15.5:0/3782733491 learned_addr learned my addr 172.21.15.5:0/3782733491 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:02.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.033+0000 7fd6f64e6640 1 --2- 172.21.15.5:0/3782733491 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd6f006d070 0x7fd6f0135c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:02.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.033+0000 7fd6f6ce7640 1 --2- 172.21.15.5:0/3782733491 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd6f00b26a0 0x7fd6f01dd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:02.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.033+0000 7fd6f5ce5640 1 -- 172.21.15.5:0/3782733491 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd6f006d070 msgr2=0x7fd6f0135c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:02.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.033+0000 7fd6f5ce5640 1 --2- 172.21.15.5:0/3782733491 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd6f006d070 0x7fd6f0135c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:02.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.033+0000 7fd6f5ce5640 1 -- 172.21.15.5:0/3782733491 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd6f00b26a0 msgr2=0x7fd6f01dd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:02.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.033+0000 7fd6f5ce5640 1 --2- 172.21.15.5:0/3782733491 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd6f00b26a0 0x7fd6f01dd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:02.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.033+0000 7fd6f5ce5640 1 -- 172.21.15.5:0/3782733491 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f0137ea0 con 0x7fd6f00af790 2024-03-18T15:34:02.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.033+0000 7fd6f6ce7640 1 --2- 172.21.15.5:0/3782733491 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd6f00b26a0 0x7fd6f01dd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:02.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.033+0000 7fd6f5ce5640 1 --2- 172.21.15.5:0/3782733491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6f00af790 0x7fd6f0144e40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd6e001ab80 tx=0x7fd6e0019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:02.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.033+0000 7fd6e6ffd640 1 -- 172.21.15.5:0/3782733491 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e00abe60 con 0x7fd6f00af790 2024-03-18T15:34:02.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.033+0000 7fd6e6ffd640 1 -- 172.21.15.5:0/3782733491 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6e00b4640 con 0x7fd6f00af790 2024-03-18T15:34:02.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.033+0000 7fd6e6ffd640 1 -- 172.21.15.5:0/3782733491 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e00b0040 con 0x7fd6f00af790 2024-03-18T15:34:02.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.033+0000 7fd6f874a640 1 -- 172.21.15.5:0/3782733491 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6f0138110 con 0x7fd6f00af790 2024-03-18T15:34:02.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.033+0000 7fd6f874a640 1 -- 172.21.15.5:0/3782733491 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6f0144050 con 0x7fd6f00af790 2024-03-18T15:34:02.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.033+0000 7fd6f874a640 1 -- 172.21.15.5:0/3782733491 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6f004f550 con 0x7fd6f00af790 2024-03-18T15:34:02.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.037+0000 7fd6e6ffd640 1 -- 172.21.15.5:0/3782733491 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fd6e00ab4c0 con 0x7fd6f00af790 2024-03-18T15:34:02.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.037+0000 7fd6e6ffd640 1 --2- 172.21.15.5:0/3782733491 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd6d4067690 0x7fd6d4069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:02.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.037+0000 7fd6e6ffd640 1 -- 172.21.15.5:0/3782733491 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd6e011f330 con 0x7fd6f00af790 2024-03-18T15:34:02.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.037+0000 7fd6f64e6640 1 --2- 172.21.15.5:0/3782733491 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd6d4067690 0x7fd6d4069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:02.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.037+0000 7fd6f64e6640 1 --2- 172.21.15.5:0/3782733491 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd6d4067690 0x7fd6d4069b50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec00ba10 tx=0x7fd6ec008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:02.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.041+0000 7fd6e6ffd640 1 -- 172.21.15.5:0/3782733491 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fd6e0016150 con 0x7fd6f00af790 2024-03-18T15:34:02.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.201+0000 7fd6f874a640 1 -- 172.21.15.5:0/3782733491 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd6f00aeba0 con 0x7fd6f00af790 2024-03-18T15:34:02.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.201+0000 7fd6e6ffd640 1 -- 172.21.15.5:0/3782733491 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7fd6e00effa0 con 0x7fd6f00af790 2024-03-18T15:34:02.204 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-03-18T15:34:02.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.205+0000 7fd6f874a640 1 -- 172.21.15.5:0/3782733491 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd6d4067690 msgr2=0x7fd6d4069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:02.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.205+0000 7fd6f874a640 1 --2- 172.21.15.5:0/3782733491 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd6d4067690 0x7fd6d4069b50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec00ba10 tx=0x7fd6ec008040 comp rx=0 tx=0).stop 2024-03-18T15:34:02.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.205+0000 7fd6f874a640 1 -- 172.21.15.5:0/3782733491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6f00af790 msgr2=0x7fd6f0144e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:02.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.205+0000 7fd6f874a640 1 --2- 172.21.15.5:0/3782733491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6f00af790 0x7fd6f0144e40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd6e001ab80 tx=0x7fd6e0019270 comp rx=0 tx=0).stop 2024-03-18T15:34:02.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.205+0000 7fd6f874a640 1 -- 172.21.15.5:0/3782733491 shutdown_connections 2024-03-18T15:34:02.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.205+0000 7fd6f874a640 1 --2- 172.21.15.5:0/3782733491 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd6d4067690 0x7fd6d4069b50 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:02.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.205+0000 7fd6f874a640 1 --2- 172.21.15.5:0/3782733491 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd6f00b26a0 0x7fd6f01dd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:02.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.205+0000 7fd6f874a640 1 --2- 172.21.15.5:0/3782733491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6f00af790 0x7fd6f0144e40 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:02.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.205+0000 7fd6f874a640 1 --2- 172.21.15.5:0/3782733491 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd6f006d070 0x7fd6f0135c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:02.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.205+0000 7fd6f874a640 1 -- 172.21.15.5:0/3782733491 >> 172.21.15.5:0/3782733491 conn(0x7fd6f00b5670 msgr2=0x7fd6f00adab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:02.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.205+0000 7fd6f874a640 1 -- 172.21.15.5:0/3782733491 shutdown_connections 2024-03-18T15:34:02.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.205+0000 7fd6f874a640 1 -- 172.21.15.5:0/3782733491 wait complete. 2024-03-18T15:34:02.224 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-18T15:34:02.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.340+0000 7fa2579bb640 1 Processor -- start 2024-03-18T15:34:02.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.340+0000 7fa2579bb640 1 -- start start 2024-03-18T15:34:02.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.340+0000 7fa2579bb640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa25014b660 0x7fa250070360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:02.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.340+0000 7fa2579bb640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa2501410d0 0x7fa250141560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:02.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.340+0000 7fa2579bb640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa25006b8c0 con 0x7fa25006b4e0 2024-03-18T15:34:02.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.340+0000 7fa2579bb640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fa25014b1d0 con 0x7fa25014b660 2024-03-18T15:34:02.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.340+0000 7fa2579bb640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fa250071630 con 0x7fa2501410d0 2024-03-18T15:34:02.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.340+0000 7fa254f56640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa25014b660 0x7fa250070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:02.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.340+0000 7fa255f58640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa2501410d0 0x7fa250141560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:02.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.340+0000 7fa255757640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fa25006b4e0 0x7fa250071da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.5:52218/0 (socket says 172.21.15.5:52218) 2024-03-18T15:34:02.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.340+0000 7fa255757640 1 -- 172.21.15.5:0/3783241908 learned_addr learned my addr 172.21.15.5:0/3783241908 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:02.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.340+0000 7fa254f56640 1 -- 172.21.15.5:0/3783241908 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa2501410d0 msgr2=0x7fa250141560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:02.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.340+0000 7fa254f56640 1 --2- 172.21.15.5:0/3783241908 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa2501410d0 0x7fa250141560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:02.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.340+0000 7fa254f56640 1 -- 172.21.15.5:0/3783241908 >> v1:172.21.15.80:6789/0 conn(0x7fa25006b4e0 legacy=0x7fa250071da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:02.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.340+0000 7fa254f56640 1 -- 172.21.15.5:0/3783241908 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa25014ab80 con 0x7fa25014b660 2024-03-18T15:34:02.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.344+0000 7fa254f56640 1 --2- 172.21.15.5:0/3783241908 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa25014b660 0x7fa250070360 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa2400097c0 tx=0x7fa24001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9c6da1d551ba550 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:02.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.344+0000 7fa247fff640 1 -- 172.21.15.5:0/3783241908 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2400b0070 con 0x7fa25014b660 2024-03-18T15:34:02.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.344+0000 7fa247fff640 1 -- 172.21.15.5:0/3783241908 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2400ab600 con 0x7fa25014b660 2024-03-18T15:34:02.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.344+0000 7fa247fff640 1 -- 172.21.15.5:0/3783241908 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2400ab9d0 con 0x7fa25014b660 2024-03-18T15:34:02.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.344+0000 7fa2579bb640 1 -- 172.21.15.5:0/3783241908 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa25014b660 msgr2=0x7fa250070360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:02.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.344+0000 7fa2579bb640 1 --2- 172.21.15.5:0/3783241908 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa25014b660 0x7fa250070360 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa2400097c0 tx=0x7fa24001c410 comp rx=0 tx=0).stop 2024-03-18T15:34:02.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.344+0000 7fa2579bb640 1 -- 172.21.15.5:0/3783241908 shutdown_connections 2024-03-18T15:34:02.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.344+0000 7fa2579bb640 1 --2- 172.21.15.5:0/3783241908 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa2501410d0 0x7fa250141560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:02.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.344+0000 7fa2579bb640 1 --2- 172.21.15.5:0/3783241908 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa25014b660 0x7fa250070360 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:02.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.344+0000 7fa2579bb640 1 -- 172.21.15.5:0/3783241908 >> 172.21.15.5:0/3783241908 conn(0x7fa2500b6680 msgr2=0x7fa2500b6a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:02.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.344+0000 7fa2579bb640 1 -- 172.21.15.5:0/3783241908 shutdown_connections 2024-03-18T15:34:02.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.344+0000 7fa2579bb640 1 -- 172.21.15.5:0/3783241908 wait complete. 2024-03-18T15:34:02.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.344+0000 7fa2579bb640 1 Processor -- start 2024-03-18T15:34:02.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.344+0000 7fa2579bb640 1 -- start start 2024-03-18T15:34:02.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.344+0000 7fa2579bb640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa25006b4e0 0x7fa250149290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:02.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.344+0000 7fa2579bb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2501410d0 0x7fa2501498d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:02.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.344+0000 7fa2579bb640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa25014b660 0x7fa250148820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:02.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.344+0000 7fa2579bb640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa250154710 con 0x7fa2501410d0 2024-03-18T15:34:02.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.344+0000 7fa2579bb640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa250154590 con 0x7fa25014b660 2024-03-18T15:34:02.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.344+0000 7fa2579bb640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fa250154890 con 0x7fa25006b4e0 2024-03-18T15:34:02.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.348+0000 7fa254f56640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2501410d0 0x7fa2501498d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:02.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.348+0000 7fa254f56640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2501410d0 0x7fa2501498d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49976/0 (socket says 172.21.15.5:49976) 2024-03-18T15:34:02.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.348+0000 7fa254f56640 1 -- 172.21.15.5:0/1222511529 learned_addr learned my addr 172.21.15.5:0/1222511529 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:02.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.348+0000 7fa255757640 1 --2- 172.21.15.5:0/1222511529 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa25006b4e0 0x7fa250149290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:02.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.348+0000 7fa255f58640 1 --2- 172.21.15.5:0/1222511529 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa25014b660 0x7fa250148820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:02.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.348+0000 7fa254f56640 1 -- 172.21.15.5:0/1222511529 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa25006b4e0 msgr2=0x7fa250149290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:02.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.348+0000 7fa254f56640 1 --2- 172.21.15.5:0/1222511529 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa25006b4e0 0x7fa250149290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:02.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.348+0000 7fa254f56640 1 -- 172.21.15.5:0/1222511529 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa25014b660 msgr2=0x7fa250148820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:02.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.348+0000 7fa254f56640 1 --2- 172.21.15.5:0/1222511529 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa25014b660 0x7fa250148820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:02.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.348+0000 7fa254f56640 1 -- 172.21.15.5:0/1222511529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2500ade10 con 0x7fa2501410d0 2024-03-18T15:34:02.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.348+0000 7fa255f58640 1 --2- 172.21.15.5:0/1222511529 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa25014b660 0x7fa250148820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:34:02.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.348+0000 7fa254f56640 1 --2- 172.21.15.5:0/1222511529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2501410d0 0x7fa2501498d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa24001a040 tx=0x7fa2400191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:02.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.348+0000 7fa245ffb640 1 -- 172.21.15.5:0/1222511529 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2400b0040 con 0x7fa2501410d0 2024-03-18T15:34:02.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.348+0000 7fa245ffb640 1 -- 172.21.15.5:0/1222511529 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2400ab400 con 0x7fa2501410d0 2024-03-18T15:34:02.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.348+0000 7fa2579bb640 1 -- 172.21.15.5:0/1222511529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa250139380 con 0x7fa2501410d0 2024-03-18T15:34:02.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.348+0000 7fa245ffb640 1 -- 172.21.15.5:0/1222511529 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa240006e20 con 0x7fa2501410d0 2024-03-18T15:34:02.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.348+0000 7fa2579bb640 1 -- 172.21.15.5:0/1222511529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2500adb30 con 0x7fa2501410d0 2024-03-18T15:34:02.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.348+0000 7fa2579bb640 1 -- 172.21.15.5:0/1222511529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa25004f550 con 0x7fa2501410d0 2024-03-18T15:34:02.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.348+0000 7fa245ffb640 1 -- 172.21.15.5:0/1222511529 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fa2400bb020 con 0x7fa2501410d0 2024-03-18T15:34:02.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.348+0000 7fa245ffb640 1 --2- 172.21.15.5:0/1222511529 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa234067660 0x7fa234069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:02.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.348+0000 7fa245ffb640 1 -- 172.21.15.5:0/1222511529 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa24011ede0 con 0x7fa2501410d0 2024-03-18T15:34:02.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.348+0000 7fa255757640 1 --2- 172.21.15.5:0/1222511529 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa234067660 0x7fa234069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:02.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.348+0000 7fa255757640 1 --2- 172.21.15.5:0/1222511529 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa234067660 0x7fa234069b20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa25013a1a0 tx=0x7fa24c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:02.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.352+0000 7fa245ffb640 1 -- 172.21.15.5:0/1222511529 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fa240016090 con 0x7fa2501410d0 2024-03-18T15:34:02.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.508+0000 7fa2579bb640 1 -- 172.21.15.5:0/1222511529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa25013fc70 con 0x7fa2501410d0 2024-03-18T15:34:02.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.508+0000 7fa245ffb640 1 -- 172.21.15.5:0/1222511529 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7fa2400efaf0 con 0x7fa2501410d0 2024-03-18T15:34:02.512 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-03-18T15:34:02.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.512+0000 7fa2579bb640 1 -- 172.21.15.5:0/1222511529 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa234067660 msgr2=0x7fa234069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:02.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.512+0000 7fa2579bb640 1 --2- 172.21.15.5:0/1222511529 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa234067660 0x7fa234069b20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa25013a1a0 tx=0x7fa24c009040 comp rx=0 tx=0).stop 2024-03-18T15:34:02.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.512+0000 7fa2579bb640 1 -- 172.21.15.5:0/1222511529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2501410d0 msgr2=0x7fa2501498d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:02.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.512+0000 7fa2579bb640 1 --2- 172.21.15.5:0/1222511529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2501410d0 0x7fa2501498d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa24001a040 tx=0x7fa2400191b0 comp rx=0 tx=0).stop 2024-03-18T15:34:02.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.512+0000 7fa2579bb640 1 -- 172.21.15.5:0/1222511529 shutdown_connections 2024-03-18T15:34:02.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.512+0000 7fa2579bb640 1 --2- 172.21.15.5:0/1222511529 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa234067660 0x7fa234069b20 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:02.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.512+0000 7fa2579bb640 1 --2- 172.21.15.5:0/1222511529 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa25014b660 0x7fa250148820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:02.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.512+0000 7fa2579bb640 1 --2- 172.21.15.5:0/1222511529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2501410d0 0x7fa2501498d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:02.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.512+0000 7fa2579bb640 1 --2- 172.21.15.5:0/1222511529 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa25006b4e0 0x7fa250149290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:02.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.512+0000 7fa2579bb640 1 -- 172.21.15.5:0/1222511529 >> 172.21.15.5:0/1222511529 conn(0x7fa2500b6680 msgr2=0x7fa2500b46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:02.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.512+0000 7fa2579bb640 1 -- 172.21.15.5:0/1222511529 shutdown_connections 2024-03-18T15:34:02.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.512+0000 7fa2579bb640 1 -- 172.21.15.5:0/1222511529 wait complete. 2024-03-18T15:34:02.530 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-18T15:34:02.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.644+0000 7f26e53af640 1 Processor -- start 2024-03-18T15:34:02.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.644+0000 7f26e53af640 1 -- start start 2024-03-18T15:34:02.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.644+0000 7f26e53af640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f26e006d2d0 0x7f26e00b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:02.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.644+0000 7f26e53af640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f26e013f260 0x7f26e013f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:02.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.644+0000 7f26e53af640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26e00b0ea0 con 0x7f26e01453a0 2024-03-18T15:34:02.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.644+0000 7f26e53af640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f26e00728c0 con 0x7f26e006d2d0 2024-03-18T15:34:02.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.644+0000 7f26e53af640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f26e006d6b0 con 0x7f26e013f260 2024-03-18T15:34:02.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.644+0000 7f26df7fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f26e01453a0 0x7f26e0145810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60048/0 (socket says 172.21.15.5:60048) 2024-03-18T15:34:02.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.644+0000 7f26df7fe640 1 -- 172.21.15.5:0/420620732 learned_addr learned my addr 172.21.15.5:0/420620732 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:02.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.644+0000 7f26deffd640 1 --2- 172.21.15.5:0/420620732 >> v2:172.21.15.80:3300/0 conn(0x7f26e006d2d0 0x7f26e00b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:02.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.644+0000 7f26de7fc640 1 --2- 172.21.15.5:0/420620732 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f26e013f260 0x7f26e013f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:02.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.648+0000 7f26deffd640 1 -- 172.21.15.5:0/420620732 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f26e013f260 msgr2=0x7f26e013f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:02.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.648+0000 7f26deffd640 1 --2- 172.21.15.5:0/420620732 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f26e013f260 0x7f26e013f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:02.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.648+0000 7f26deffd640 1 -- 172.21.15.5:0/420620732 >> v1:172.21.15.5:6789/0 conn(0x7f26e01453a0 legacy=0x7f26e0145810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:02.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.648+0000 7f26ddffb640 1 -- 172.21.15.5:0/420620732 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4239323133 0 0) 0x7f26e00b0ea0 con 0x7f26e01453a0 2024-03-18T15:34:02.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.648+0000 7f26deffd640 1 -- 172.21.15.5:0/420620732 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e00b0bc0 con 0x7f26e006d2d0 2024-03-18T15:34:02.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.648+0000 7f26deffd640 1 --2- 172.21.15.5:0/420620732 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26e006d2d0 0x7f26e00b23a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f26d8009d90 tx=0x7f26d801c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b70d9330dc535b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:02.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.648+0000 7f26ddffb640 1 -- 172.21.15.5:0/420620732 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d80b2070 con 0x7f26e006d2d0 2024-03-18T15:34:02.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.648+0000 7f26ddffb640 1 -- 172.21.15.5:0/420620732 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26d80a58b0 con 0x7f26e006d2d0 2024-03-18T15:34:02.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.648+0000 7f26ddffb640 1 -- 172.21.15.5:0/420620732 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d80a5c80 con 0x7f26e006d2d0 2024-03-18T15:34:02.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.648+0000 7f26e53af640 1 -- 172.21.15.5:0/420620732 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26e006d2d0 msgr2=0x7f26e00b23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:02.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.648+0000 7f26e53af640 1 --2- 172.21.15.5:0/420620732 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26e006d2d0 0x7f26e00b23a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f26d8009d90 tx=0x7f26d801c8e0 comp rx=0 tx=0).stop 2024-03-18T15:34:02.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.648+0000 7f26e53af640 1 -- 172.21.15.5:0/420620732 shutdown_connections 2024-03-18T15:34:02.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.648+0000 7f26e53af640 1 --2- 172.21.15.5:0/420620732 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f26e013f260 0x7f26e013f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:02.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.648+0000 7f26e53af640 1 --2- 172.21.15.5:0/420620732 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26e006d2d0 0x7f26e00b23a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:02.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.648+0000 7f26e53af640 1 -- 172.21.15.5:0/420620732 >> 172.21.15.5:0/420620732 conn(0x7f26e00b5670 msgr2=0x7f26e00b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:02.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.648+0000 7f26e53af640 1 -- 172.21.15.5:0/420620732 shutdown_connections 2024-03-18T15:34:02.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.648+0000 7f26e53af640 1 -- 172.21.15.5:0/420620732 wait complete. 2024-03-18T15:34:02.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.652+0000 7f26e53af640 1 Processor -- start 2024-03-18T15:34:02.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.652+0000 7f26e53af640 1 -- start start 2024-03-18T15:34:02.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.652+0000 7f26e53af640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26e006d2d0 0x7f26e00716d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:02.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.652+0000 7f26e53af640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f26e013f260 0x7f26e0141db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:02.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.652+0000 7f26e53af640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26e01453a0 0x7f26e01368b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:02.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.652+0000 7f26e53af640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f26e0072ed0 con 0x7f26e01453a0 2024-03-18T15:34:02.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.652+0000 7f26e53af640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f26e0072d50 con 0x7f26e006d2d0 2024-03-18T15:34:02.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.652+0000 7f26e53af640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f26e00b5140 con 0x7f26e013f260 2024-03-18T15:34:02.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.652+0000 7f26df7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26e01453a0 0x7f26e01368b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:02.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.652+0000 7f26df7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26e01453a0 0x7f26e01368b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49986/0 (socket says 172.21.15.5:49986) 2024-03-18T15:34:02.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.652+0000 7f26df7fe640 1 -- 172.21.15.5:0/2557951074 learned_addr learned my addr 172.21.15.5:0/2557951074 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:02.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.652+0000 7f26deffd640 1 --2- 172.21.15.5:0/2557951074 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26e006d2d0 0x7f26e00716d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:02.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.652+0000 7f26de7fc640 1 --2- 172.21.15.5:0/2557951074 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f26e013f260 0x7f26e0141db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:02.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.652+0000 7f26df7fe640 1 -- 172.21.15.5:0/2557951074 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f26e013f260 msgr2=0x7f26e0141db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:02.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.652+0000 7f26df7fe640 1 --2- 172.21.15.5:0/2557951074 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f26e013f260 0x7f26e0141db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:02.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.652+0000 7f26df7fe640 1 -- 172.21.15.5:0/2557951074 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26e006d2d0 msgr2=0x7f26e00716d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:02.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.652+0000 7f26df7fe640 1 --2- 172.21.15.5:0/2557951074 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26e006d2d0 0x7f26e00716d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:02.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.652+0000 7f26df7fe640 1 -- 172.21.15.5:0/2557951074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e013feb0 con 0x7f26e01453a0 2024-03-18T15:34:02.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.652+0000 7f26deffd640 1 --2- 172.21.15.5:0/2557951074 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26e006d2d0 0x7f26e00716d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:34:02.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.652+0000 7f26df7fe640 1 --2- 172.21.15.5:0/2557951074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26e01453a0 0x7f26e01368b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f26d000c480 tx=0x7f26d000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:02.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.652+0000 7f26d77fe640 1 -- 172.21.15.5:0/2557951074 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d0018070 con 0x7f26e01453a0 2024-03-18T15:34:02.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.652+0000 7f26d77fe640 1 -- 172.21.15.5:0/2557951074 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26d00074e0 con 0x7f26e01453a0 2024-03-18T15:34:02.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.652+0000 7f26e53af640 1 -- 172.21.15.5:0/2557951074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26e0140190 con 0x7f26e01453a0 2024-03-18T15:34:02.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.652+0000 7f26d77fe640 1 -- 172.21.15.5:0/2557951074 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d0004a30 con 0x7f26e01453a0 2024-03-18T15:34:02.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.652+0000 7f26e53af640 1 -- 172.21.15.5:0/2557951074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26e0149560 con 0x7f26e01453a0 2024-03-18T15:34:02.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.652+0000 7f26e53af640 1 -- 172.21.15.5:0/2557951074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26e004f550 con 0x7f26e01453a0 2024-03-18T15:34:02.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.652+0000 7f26d77fe640 1 -- 172.21.15.5:0/2557951074 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f26d001b070 con 0x7f26e01453a0 2024-03-18T15:34:02.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.652+0000 7f26d77fe640 1 --2- 172.21.15.5:0/2557951074 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f26c0067660 0x7f26c0069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:02.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.652+0000 7f26d77fe640 1 -- 172.21.15.5:0/2557951074 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f26d0084b30 con 0x7f26e01453a0 2024-03-18T15:34:02.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.652+0000 7f26deffd640 1 --2- 172.21.15.5:0/2557951074 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f26c0067660 0x7f26c0069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:02.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.656+0000 7f26deffd640 1 --2- 172.21.15.5:0/2557951074 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f26c0067660 0x7f26c0069b20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f26d80af000 tx=0x7f26d80a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:02.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.660+0000 7f26d77fe640 1 -- 172.21.15.5:0/2557951074 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f26d0055840 con 0x7f26e01453a0 2024-03-18T15:34:02.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.812+0000 7f26e53af640 1 -- 172.21.15.5:0/2557951074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f26e006b990 con 0x7f26e01453a0 2024-03-18T15:34:02.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.812+0000 7f26d77fe640 1 -- 172.21.15.5:0/2557951074 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f26d005b020 con 0x7f26e01453a0 2024-03-18T15:34:02.816 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-03-18T15:34:02.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.816+0000 7f26e53af640 1 -- 172.21.15.5:0/2557951074 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f26c0067660 msgr2=0x7f26c0069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:02.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.816+0000 7f26e53af640 1 --2- 172.21.15.5:0/2557951074 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f26c0067660 0x7f26c0069b20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f26d80af000 tx=0x7f26d80a7070 comp rx=0 tx=0).stop 2024-03-18T15:34:02.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.816+0000 7f26e53af640 1 -- 172.21.15.5:0/2557951074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26e01453a0 msgr2=0x7f26e01368b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:02.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.816+0000 7f26e53af640 1 --2- 172.21.15.5:0/2557951074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26e01453a0 0x7f26e01368b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f26d000c480 tx=0x7f26d000c940 comp rx=0 tx=0).stop 2024-03-18T15:34:02.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.816+0000 7f26e53af640 1 -- 172.21.15.5:0/2557951074 shutdown_connections 2024-03-18T15:34:02.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.816+0000 7f26e53af640 1 --2- 172.21.15.5:0/2557951074 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f26c0067660 0x7f26c0069b20 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:02.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.816+0000 7f26e53af640 1 --2- 172.21.15.5:0/2557951074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f26e01453a0 0x7f26e01368b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:02.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.816+0000 7f26e53af640 1 --2- 172.21.15.5:0/2557951074 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f26e013f260 0x7f26e0141db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:02.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.816+0000 7f26e53af640 1 --2- 172.21.15.5:0/2557951074 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26e006d2d0 0x7f26e00716d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:02.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.816+0000 7f26e53af640 1 -- 172.21.15.5:0/2557951074 >> 172.21.15.5:0/2557951074 conn(0x7f26e00b5670 msgr2=0x7f26e00b5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:02.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.816+0000 7f26e53af640 1 -- 172.21.15.5:0/2557951074 shutdown_connections 2024-03-18T15:34:02.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.816+0000 7f26e53af640 1 -- 172.21.15.5:0/2557951074 wait complete. 2024-03-18T15:34:02.859 INFO:tasks.cephfs_test_runner:Starting test: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD) Test that when cd is issued with an argument, CWD is changed 2024-03-18T15:34:02.859 INFO:tasks.cephfs_test_runner:test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD) 2024-03-18T15:34:02.859 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:34:02.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.968+0000 7fd731f7c640 1 Processor -- start 2024-03-18T15:34:02.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.968+0000 7fd731f7c640 1 -- start start 2024-03-18T15:34:02.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.968+0000 7fd731f7c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd72c0b6d40 0x7fd72c148a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:02.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.968+0000 7fd731f7c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd72c148f90 0x7fd72c0b4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:02.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.968+0000 7fd731f7c640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd72c149420 con 0x7fd72c0b4c70 2024-03-18T15:34:02.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.968+0000 7fd731f7c640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fd72c0b78e0 con 0x7fd72c0b6d40 2024-03-18T15:34:02.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.968+0000 7fd731f7c640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fd72c0b7120 con 0x7fd72c148f90 2024-03-18T15:34:02.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.968+0000 7fd72b7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd72c0b6d40 0x7fd72c148a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:02.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.968+0000 7fd72bfff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd72c0b4c70 0x7fd72c071380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60062/0 (socket says 172.21.15.5:60062) 2024-03-18T15:34:02.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.968+0000 7fd72b7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd72c0b6d40 0x7fd72c148a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49992/0 (socket says 172.21.15.5:49992) 2024-03-18T15:34:02.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.968+0000 7fd72bfff640 1 -- 172.21.15.5:0/680042036 learned_addr learned my addr 172.21.15.5:0/680042036 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:02.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.968+0000 7fd72affd640 1 --2- 172.21.15.5:0/680042036 >> v2:172.21.15.80:3300/0 conn(0x7fd72c148f90 0x7fd72c0b4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:02.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.968+0000 7fd72b7fe640 1 -- 172.21.15.5:0/680042036 >> v2:172.21.15.80:3300/0 conn(0x7fd72c148f90 msgr2=0x7fd72c0b4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:02.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.968+0000 7fd72b7fe640 1 --2- 172.21.15.5:0/680042036 >> v2:172.21.15.80:3300/0 conn(0x7fd72c148f90 0x7fd72c0b4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:02.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.968+0000 7fd72b7fe640 1 -- 172.21.15.5:0/680042036 >> v1:172.21.15.5:6789/0 conn(0x7fd72c0b4c70 legacy=0x7fd72c071380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:02.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.968+0000 7fd72a7fc640 1 -- 172.21.15.5:0/680042036 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3884487520 0 0) 0x7fd72c149420 con 0x7fd72c0b4c70 2024-03-18T15:34:02.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.968+0000 7fd72b7fe640 1 -- 172.21.15.5:0/680042036 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd72c0b5e20 con 0x7fd72c0b6d40 2024-03-18T15:34:02.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.972+0000 7fd72b7fe640 1 --2- 172.21.15.5:0/680042036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd72c0b6d40 0x7fd72c148a50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd72c0711d0 tx=0x7fd71c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cab942ab66ae6882 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:02.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.972+0000 7fd72a7fc640 1 -- 172.21.15.5:0/680042036 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71c0b0070 con 0x7fd72c0b6d40 2024-03-18T15:34:02.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.972+0000 7fd72a7fc640 1 -- 172.21.15.5:0/680042036 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd71c0ab600 con 0x7fd72c0b6d40 2024-03-18T15:34:02.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.972+0000 7fd72a7fc640 1 -- 172.21.15.5:0/680042036 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71c0ab9d0 con 0x7fd72c0b6d40 2024-03-18T15:34:02.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.972+0000 7fd731f7c640 1 -- 172.21.15.5:0/680042036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd72c0b6d40 msgr2=0x7fd72c148a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:02.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.972+0000 7fd731f7c640 1 --2- 172.21.15.5:0/680042036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd72c0b6d40 0x7fd72c148a50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd72c0711d0 tx=0x7fd71c01c3f0 comp rx=0 tx=0).stop 2024-03-18T15:34:02.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.972+0000 7fd731f7c640 1 -- 172.21.15.5:0/680042036 shutdown_connections 2024-03-18T15:34:02.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.972+0000 7fd731f7c640 1 --2- 172.21.15.5:0/680042036 >> v2:172.21.15.80:3300/0 conn(0x7fd72c148f90 0x7fd72c0b4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:02.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.972+0000 7fd731f7c640 1 --2- 172.21.15.5:0/680042036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd72c0b6d40 0x7fd72c148a50 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:02.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.972+0000 7fd731f7c640 1 -- 172.21.15.5:0/680042036 >> 172.21.15.5:0/680042036 conn(0x7fd72c066fc0 msgr2=0x7fd72c0673d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:02.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.972+0000 7fd731f7c640 1 -- 172.21.15.5:0/680042036 shutdown_connections 2024-03-18T15:34:02.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.972+0000 7fd731f7c640 1 -- 172.21.15.5:0/680042036 wait complete. 2024-03-18T15:34:02.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.972+0000 7fd731f7c640 1 Processor -- start 2024-03-18T15:34:02.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.972+0000 7fd731f7c640 1 -- start start 2024-03-18T15:34:02.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.972+0000 7fd731f7c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd72c0b4c70 0x7fd72c06dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:02.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.972+0000 7fd731f7c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd72c0b6d40 0x7fd72c1501f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:02.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.972+0000 7fd731f7c640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd72c148f90 0x7fd72c14f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:02.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.972+0000 7fd731f7c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd72c152e20 con 0x7fd72c0b6d40 2024-03-18T15:34:02.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.972+0000 7fd731f7c640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd72c152ca0 con 0x7fd72c0b4c70 2024-03-18T15:34:02.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.972+0000 7fd731f7c640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fd72c152fa0 con 0x7fd72c148f90 2024-03-18T15:34:02.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.972+0000 7fd72affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd72c0b6d40 0x7fd72c1501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:02.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.972+0000 7fd72affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd72c0b6d40 0x7fd72c1501f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50008/0 (socket says 172.21.15.5:50008) 2024-03-18T15:34:02.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.972+0000 7fd72affd640 1 -- 172.21.15.5:0/1671456229 learned_addr learned my addr 172.21.15.5:0/1671456229 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:02.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.972+0000 7fd72b7fe640 1 --2- 172.21.15.5:0/1671456229 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd72c0b4c70 0x7fd72c06dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:02.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.972+0000 7fd72bfff640 1 --2- 172.21.15.5:0/1671456229 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd72c148f90 0x7fd72c14f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:02.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.972+0000 7fd72affd640 1 -- 172.21.15.5:0/1671456229 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd72c148f90 msgr2=0x7fd72c14f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:02.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.972+0000 7fd72affd640 1 --2- 172.21.15.5:0/1671456229 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd72c148f90 0x7fd72c14f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:02.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.972+0000 7fd72affd640 1 -- 172.21.15.5:0/1671456229 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd72c0b4c70 msgr2=0x7fd72c06dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:02.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.972+0000 7fd72affd640 1 --2- 172.21.15.5:0/1671456229 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd72c0b4c70 0x7fd72c06dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:02.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.972+0000 7fd72affd640 1 -- 172.21.15.5:0/1671456229 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd72c0b1720 con 0x7fd72c0b6d40 2024-03-18T15:34:02.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.972+0000 7fd72affd640 1 --2- 172.21.15.5:0/1671456229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd72c0b6d40 0x7fd72c1501f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd714007e60 tx=0x7fd71400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:02.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.972+0000 7fd723fff640 1 -- 172.21.15.5:0/1671456229 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd714014620 con 0x7fd72c0b6d40 2024-03-18T15:34:02.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.972+0000 7fd723fff640 1 -- 172.21.15.5:0/1671456229 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd714019070 con 0x7fd72c0b6d40 2024-03-18T15:34:02.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.972+0000 7fd731f7c640 1 -- 172.21.15.5:0/1671456229 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd72c140010 con 0x7fd72c0b6d40 2024-03-18T15:34:02.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.972+0000 7fd723fff640 1 -- 172.21.15.5:0/1671456229 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd714014a20 con 0x7fd72c0b6d40 2024-03-18T15:34:02.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.972+0000 7fd731f7c640 1 -- 172.21.15.5:0/1671456229 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd72c140780 con 0x7fd72c0b6d40 2024-03-18T15:34:02.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.976+0000 7fd731f7c640 1 -- 172.21.15.5:0/1671456229 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd72c14e150 con 0x7fd72c0b6d40 2024-03-18T15:34:02.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.976+0000 7fd723fff640 1 -- 172.21.15.5:0/1671456229 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fd714014bc0 con 0x7fd72c0b6d40 2024-03-18T15:34:02.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.976+0000 7fd723fff640 1 --2- 172.21.15.5:0/1671456229 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd70c067660 0x7fd70c069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:02.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.976+0000 7fd723fff640 1 -- 172.21.15.5:0/1671456229 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd714086100 con 0x7fd72c0b6d40 2024-03-18T15:34:02.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.976+0000 7fd72b7fe640 1 --2- 172.21.15.5:0/1671456229 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd70c067660 0x7fd70c069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:02.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.976+0000 7fd72b7fe640 1 --2- 172.21.15.5:0/1671456229 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd70c067660 0x7fd70c069b20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd71c0ad000 tx=0x7fd71c0a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:02.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:02.980+0000 7fd723fff640 1 -- 172.21.15.5:0/1671456229 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fd714056e10 con 0x7fd72c0b6d40 2024-03-18T15:34:03.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.136+0000 7fd731f7c640 1 -- 172.21.15.5:0/1671456229 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd72c0b8ec0 con 0x7fd72c0b6d40 2024-03-18T15:34:03.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.140+0000 7fd723fff640 1 -- 172.21.15.5:0/1671456229 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16449 (secure 0 0 0) 0x7fd714056710 con 0x7fd72c0b6d40 2024-03-18T15:34:03.142 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:34:03.142 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":21,"fsid":"ef54fe03-5aa7-4eb2-9b1e-66c9cc345b94","created":"2024-03-18T15:33:18.086614+0000","modified":"2024-03-18T15:33:43.984050+0000","last_up_change":"2024-03-18T15:33:29.229911+0000","last_in_change":"2024-03-18T15:33:20.323840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:28.963186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T15:33:40.533646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T15:33:41.323519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df1c754b-af59-4d16-8919-fbb6386b3c48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6804","nonce":976264754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6808","nonce":976264754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6815","nonce":976264754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6811","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6813","nonce":976264754}]},"public_addr":"172.21.15.5:6804/976264754","cluster_addr":"172.21.15.5:6808/976264754","heartbeat_back_addr":"172.21.15.5:6815/976264754","heartbeat_front_addr":"172.21.15.5:6813/976264754","state":["exists","up"]},{"osd":1,"uuid":"b9a08b5e-8af4-4144-bdba-d69ed1ac627a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6801","nonce":3901761126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6803","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6805","nonce":3901761126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6812","nonce":3901761126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6807","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6809","nonce":3901761126}]},"public_addr":"172.21.15.5:6801/3901761126","cluster_addr":"172.21.15.5:6805/3901761126","heartbeat_back_addr":"172.21.15.5:6812/3901761126","heartbeat_front_addr":"172.21.15.5:6809/3901761126","state":["exists","up"]},{"osd":2,"uuid":"c3e8bb76-e382-439d-b380-62ef92f1255a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6817","nonce":502968448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6819","nonce":502968448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6823","nonce":502968448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6821","nonce":502968448}]},"public_addr":"172.21.15.5:6817/502968448","cluster_addr":"172.21.15.5:6819/502968448","heartbeat_back_addr":"172.21.15.5:6823/502968448","heartbeat_front_addr":"172.21.15.5:6821/502968448","state":["exists","up"]},{"osd":3,"uuid":"e8c94625-f2cd-4eba-b53d-2e7356e1c02a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6825","nonce":2425932735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6827","nonce":2425932735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6831","nonce":2425932735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6829","nonce":2425932735}]},"public_addr":"172.21.15.5:6825/2425932735","cluster_addr":"172.21.15.5:6827/2425932735","heartbeat_back_addr":"172.21.15.5:6831/2425932735","heartbeat_front_addr":"172.21.15.5:6829/2425932735","state":["exists","up"]},{"osd":4,"uuid":"0099dd52-6265-4d32-82e4-3ef16e73add0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6817","nonce":976514285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6819","nonce":976514285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6823","nonce":976514285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6821","nonce":976514285}]},"public_addr":"172.21.15.80:6817/976514285","cluster_addr":"172.21.15.80:6819/976514285","heartbeat_back_addr":"172.21.15.80:6823/976514285","heartbeat_front_addr":"172.21.15.80:6821/976514285","state":["exists","up"]},{"osd":5,"uuid":"ed10c84f-8c39-41cc-9b64-7c482a94a48e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6825","nonce":627153447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6827","nonce":627153447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6831","nonce":627153447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6829","nonce":627153447}]},"public_addr":"172.21.15.80:6825/627153447","cluster_addr":"172.21.15.80:6827/627153447","heartbeat_back_addr":"172.21.15.80:6831/627153447","heartbeat_front_addr":"172.21.15.80:6829/627153447","state":["exists","up"]},{"osd":6,"uuid":"c7a3538a-7048-4fba-8781-b440f03c25db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6803","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6805","nonce":488342527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6807","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6809","nonce":488342527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6815","nonce":488342527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6811","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6813","nonce":488342527}]},"public_addr":"172.21.15.80:6805/488342527","cluster_addr":"172.21.15.80:6809/488342527","heartbeat_back_addr":"172.21.15.80:6815/488342527","heartbeat_front_addr":"172.21.15.80:6813/488342527","state":["exists","up"]},{"osd":7,"uuid":"f9418104-3a97-4110-bc3a-1f8151d0f757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6801","nonce":982831666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6804","nonce":982831666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6812","nonce":982831666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6808","nonce":982831666}]},"public_addr":"172.21.15.80:6801/982831666","cluster_addr":"172.21.15.80:6804/982831666","heartbeat_back_addr":"172.21.15.80:6812/982831666","heartbeat_front_addr":"172.21.15.80:6808/982831666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.970828+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.858116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.987556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.864603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.756293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.591502+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:34:03.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.140+0000 7fd731f7c640 1 -- 172.21.15.5:0/1671456229 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd70c067660 msgr2=0x7fd70c069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:03.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.140+0000 7fd731f7c640 1 --2- 172.21.15.5:0/1671456229 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd70c067660 0x7fd70c069b20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd71c0ad000 tx=0x7fd71c0a6000 comp rx=0 tx=0).stop 2024-03-18T15:34:03.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.140+0000 7fd731f7c640 1 -- 172.21.15.5:0/1671456229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd72c0b6d40 msgr2=0x7fd72c1501f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:03.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.140+0000 7fd731f7c640 1 --2- 172.21.15.5:0/1671456229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd72c0b6d40 0x7fd72c1501f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd714007e60 tx=0x7fd71400e4b0 comp rx=0 tx=0).stop 2024-03-18T15:34:03.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.140+0000 7fd731f7c640 1 -- 172.21.15.5:0/1671456229 shutdown_connections 2024-03-18T15:34:03.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.140+0000 7fd731f7c640 1 --2- 172.21.15.5:0/1671456229 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd70c067660 0x7fd70c069b20 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:03.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.140+0000 7fd731f7c640 1 --2- 172.21.15.5:0/1671456229 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd72c148f90 0x7fd72c14f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:03.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.140+0000 7fd731f7c640 1 --2- 172.21.15.5:0/1671456229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd72c0b6d40 0x7fd72c1501f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:03.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.140+0000 7fd731f7c640 1 --2- 172.21.15.5:0/1671456229 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd72c0b4c70 0x7fd72c06dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:03.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.140+0000 7fd731f7c640 1 -- 172.21.15.5:0/1671456229 >> 172.21.15.5:0/1671456229 conn(0x7fd72c066fc0 msgr2=0x7fd72c13f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:03.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.140+0000 7fd731f7c640 1 -- 172.21.15.5:0/1671456229 shutdown_connections 2024-03-18T15:34:03.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.140+0000 7fd731f7c640 1 -- 172.21.15.5:0/1671456229 wait complete. 2024-03-18T15:34:03.160 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T15:33:28.963186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-18T15:33:40.533646+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-18T15:33:41.323519+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T15:34:03.160 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T15:34:03.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.272+0000 7fd1132de640 1 Processor -- start 2024-03-18T15:34:03.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.272+0000 7fd1132de640 1 -- start start 2024-03-18T15:34:03.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.272+0000 7fd1132de640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd10c0b1bc0 0x7fd10c06d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:03.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.272+0000 7fd1132de640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd10c141b20 0x7fd10c0b6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:03.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.272+0000 7fd1132de640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd10c0b7d80 con 0x7fd10c0b79a0 2024-03-18T15:34:03.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.272+0000 7fd1132de640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fd10c0b6250 con 0x7fd10c141b20 2024-03-18T15:34:03.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.272+0000 7fd1132de640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fd10c0b6530 con 0x7fd10c0b1bc0 2024-03-18T15:34:03.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.272+0000 7fd11187b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd10c141b20 0x7fd10c0b6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:03.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.272+0000 7fd11187b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd10c141b20 0x7fd10c0b6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50016/0 (socket says 172.21.15.5:50016) 2024-03-18T15:34:03.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.272+0000 7fd11187b640 1 -- 172.21.15.5:0/2154425376 learned_addr learned my addr 172.21.15.5:0/2154425376 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:03.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.272+0000 7fd110879640 1 --2- 172.21.15.5:0/2154425376 >> v2:172.21.15.80:3300/0 conn(0x7fd10c0b1bc0 0x7fd10c06d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:03.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.272+0000 7fd11187b640 1 -- 172.21.15.5:0/2154425376 >> v2:172.21.15.80:3300/0 conn(0x7fd10c0b1bc0 msgr2=0x7fd10c06d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:03.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.272+0000 7fd11187b640 1 --2- 172.21.15.5:0/2154425376 >> v2:172.21.15.80:3300/0 conn(0x7fd10c0b1bc0 0x7fd10c06d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:03.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.272+0000 7fd11187b640 1 -- 172.21.15.5:0/2154425376 >> v1:172.21.15.80:6789/0 conn(0x7fd10c0b79a0 legacy=0x7fd10c141410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:03.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.272+0000 7fd11187b640 1 -- 172.21.15.5:0/2154425376 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd10c0b56b0 con 0x7fd10c141b20 2024-03-18T15:34:03.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.276+0000 7fd11187b640 1 --2- 172.21.15.5:0/2154425376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd10c141b20 0x7fd10c0b6cf0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd10c0b8030 tx=0x7fd10401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a1315084d47ca5b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:03.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.276+0000 7fd103fff640 1 -- 172.21.15.5:0/2154425376 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1040a57d0 con 0x7fd10c141b20 2024-03-18T15:34:03.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.276+0000 7fd103fff640 1 -- 172.21.15.5:0/2154425376 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1040a5970 con 0x7fd10c141b20 2024-03-18T15:34:03.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.276+0000 7fd103fff640 1 -- 172.21.15.5:0/2154425376 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1040a5d40 con 0x7fd10c141b20 2024-03-18T15:34:03.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.276+0000 7fd1132de640 1 -- 172.21.15.5:0/2154425376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd10c141b20 msgr2=0x7fd10c0b6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:03.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.276+0000 7fd1132de640 1 --2- 172.21.15.5:0/2154425376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd10c141b20 0x7fd10c0b6cf0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fd10c0b8030 tx=0x7fd10401c9f0 comp rx=0 tx=0).stop 2024-03-18T15:34:03.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.276+0000 7fd1132de640 1 -- 172.21.15.5:0/2154425376 shutdown_connections 2024-03-18T15:34:03.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.276+0000 7fd1132de640 1 --2- 172.21.15.5:0/2154425376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd10c141b20 0x7fd10c0b6cf0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:03.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.276+0000 7fd1132de640 1 --2- 172.21.15.5:0/2154425376 >> v2:172.21.15.80:3300/0 conn(0x7fd10c0b1bc0 0x7fd10c06d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:03.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.276+0000 7fd1132de640 1 -- 172.21.15.5:0/2154425376 >> 172.21.15.5:0/2154425376 conn(0x7fd10c066fc0 msgr2=0x7fd10c0673d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:03.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.276+0000 7fd1132de640 1 -- 172.21.15.5:0/2154425376 shutdown_connections 2024-03-18T15:34:03.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.276+0000 7fd1132de640 1 -- 172.21.15.5:0/2154425376 wait complete. 2024-03-18T15:34:03.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.276+0000 7fd1132de640 1 Processor -- start 2024-03-18T15:34:03.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.276+0000 7fd1132de640 1 -- start start 2024-03-18T15:34:03.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.276+0000 7fd1132de640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd10c0b79a0 0x7fd10c13bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:03.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.276+0000 7fd1132de640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd10c141b20 0x7fd10c13ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:03.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.276+0000 7fd1132de640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd10c13b350 0x7fd10c0b3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:03.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.276+0000 7fd1132de640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd10c1548b0 con 0x7fd10c141b20 2024-03-18T15:34:03.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.276+0000 7fd1132de640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd10c072cf0 con 0x7fd10c13b350 2024-03-18T15:34:03.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.276+0000 7fd1132de640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fd10c154a30 con 0x7fd10c0b79a0 2024-03-18T15:34:03.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.276+0000 7fd110879640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd10c141b20 0x7fd10c13ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:03.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.276+0000 7fd110879640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd10c141b20 0x7fd10c13ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50026/0 (socket says 172.21.15.5:50026) 2024-03-18T15:34:03.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.276+0000 7fd110879640 1 -- 172.21.15.5:0/1764334737 learned_addr learned my addr 172.21.15.5:0/1764334737 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:03.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.276+0000 7fd11187b640 1 --2- 172.21.15.5:0/1764334737 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd10c13b350 0x7fd10c0b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:03.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.276+0000 7fd11107a640 1 --2- 172.21.15.5:0/1764334737 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd10c0b79a0 0x7fd10c13bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:03.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.276+0000 7fd110879640 1 -- 172.21.15.5:0/1764334737 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd10c0b79a0 msgr2=0x7fd10c13bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:03.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.276+0000 7fd110879640 1 --2- 172.21.15.5:0/1764334737 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd10c0b79a0 0x7fd10c13bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:03.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.276+0000 7fd110879640 1 -- 172.21.15.5:0/1764334737 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd10c13b350 msgr2=0x7fd10c0b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:03.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.276+0000 7fd110879640 1 --2- 172.21.15.5:0/1764334737 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd10c13b350 0x7fd10c0b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:03.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.276+0000 7fd110879640 1 -- 172.21.15.5:0/1764334737 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd10c1420f0 con 0x7fd10c141b20 2024-03-18T15:34:03.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.276+0000 7fd110879640 1 --2- 172.21.15.5:0/1764334737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd10c141b20 0x7fd10c13ae10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc00e980 tx=0x7fd0fc00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:03.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.280+0000 7fd101ffb640 1 -- 172.21.15.5:0/1764334737 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc017070 con 0x7fd10c141b20 2024-03-18T15:34:03.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.280+0000 7fd101ffb640 1 -- 172.21.15.5:0/1764334737 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0fc004020 con 0x7fd10c141b20 2024-03-18T15:34:03.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.280+0000 7fd101ffb640 1 -- 172.21.15.5:0/1764334737 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0fc0043f0 con 0x7fd10c141b20 2024-03-18T15:34:03.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.280+0000 7fd1132de640 1 -- 172.21.15.5:0/1764334737 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd10c13b7d0 con 0x7fd10c141b20 2024-03-18T15:34:03.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.280+0000 7fd1132de640 1 -- 172.21.15.5:0/1764334737 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd10c1e3280 con 0x7fd10c141b20 2024-03-18T15:34:03.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.280+0000 7fd1132de640 1 -- 172.21.15.5:0/1764334737 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd10c04f550 con 0x7fd10c141b20 2024-03-18T15:34:03.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.280+0000 7fd101ffb640 1 -- 172.21.15.5:0/1764334737 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fd0fc00c9a0 con 0x7fd10c141b20 2024-03-18T15:34:03.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.280+0000 7fd101ffb640 1 --2- 172.21.15.5:0/1764334737 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd0f0067690 0x7fd0f0069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:03.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.280+0000 7fd101ffb640 1 -- 172.21.15.5:0/1764334737 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd0fc0857e0 con 0x7fd10c141b20 2024-03-18T15:34:03.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.280+0000 7fd11107a640 1 --2- 172.21.15.5:0/1764334737 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd0f0067690 0x7fd0f0069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:03.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.280+0000 7fd11107a640 1 --2- 172.21.15.5:0/1764334737 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd0f0067690 0x7fd0f0069b50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd108007fd0 tx=0x7fd108009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:03.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.284+0000 7fd101ffb640 1 -- 172.21.15.5:0/1764334737 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fd10c13b7d0 con 0x7fd10c141b20 2024-03-18T15:34:03.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.436+0000 7fd1132de640 1 -- 172.21.15.5:0/1764334737 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd10c1e39f0 con 0x7fd10c141b20 2024-03-18T15:34:03.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.440+0000 7fd101ffb640 1 -- 172.21.15.5:0/1764334737 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7fd0fc0564f0 con 0x7fd10c141b20 2024-03-18T15:34:03.441 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-03-18T15:34:03.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.440+0000 7fd1132de640 1 -- 172.21.15.5:0/1764334737 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd0f0067690 msgr2=0x7fd0f0069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:03.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.440+0000 7fd1132de640 1 --2- 172.21.15.5:0/1764334737 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd0f0067690 0x7fd0f0069b50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd108007fd0 tx=0x7fd108009040 comp rx=0 tx=0).stop 2024-03-18T15:34:03.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.440+0000 7fd1132de640 1 -- 172.21.15.5:0/1764334737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd10c141b20 msgr2=0x7fd10c13ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:03.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.440+0000 7fd1132de640 1 --2- 172.21.15.5:0/1764334737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd10c141b20 0x7fd10c13ae10 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc00e980 tx=0x7fd0fc00ee40 comp rx=0 tx=0).stop 2024-03-18T15:34:03.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.440+0000 7fd1132de640 1 -- 172.21.15.5:0/1764334737 shutdown_connections 2024-03-18T15:34:03.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.440+0000 7fd1132de640 1 --2- 172.21.15.5:0/1764334737 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd0f0067690 0x7fd0f0069b50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:03.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.440+0000 7fd1132de640 1 --2- 172.21.15.5:0/1764334737 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd10c13b350 0x7fd10c0b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:03.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.440+0000 7fd1132de640 1 --2- 172.21.15.5:0/1764334737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd10c141b20 0x7fd10c13ae10 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:03.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.440+0000 7fd1132de640 1 --2- 172.21.15.5:0/1764334737 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd10c0b79a0 0x7fd10c13bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:03.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.440+0000 7fd1132de640 1 -- 172.21.15.5:0/1764334737 >> 172.21.15.5:0/1764334737 conn(0x7fd10c066fc0 msgr2=0x7fd10c13ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:03.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.440+0000 7fd1132de640 1 -- 172.21.15.5:0/1764334737 shutdown_connections 2024-03-18T15:34:03.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.440+0000 7fd1132de640 1 -- 172.21.15.5:0/1764334737 wait complete. 2024-03-18T15:34:03.460 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-18T15:34:03.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.572+0000 7f2af08bf640 1 Processor -- start 2024-03-18T15:34:03.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.572+0000 7f2af08bf640 1 -- start start 2024-03-18T15:34:03.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.572+0000 7f2af08bf640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2aec072b40 0x7f2aec06d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:03.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.572+0000 7f2af08bf640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2aec0afb70 con 0x7f2aec0af790 2024-03-18T15:34:03.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.572+0000 7f2af08bf640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2aec0b4a20 con 0x7f2aec0b26a0 2024-03-18T15:34:03.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.572+0000 7f2af08bf640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f2aec06d5b0 con 0x7f2aec072b40 2024-03-18T15:34:03.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.572+0000 7f2ae9d9b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f2aec072b40 0x7f2aec06d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:03.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.572+0000 7f2aea59c640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f2aec0af790 0x7f2aec147780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60064/0 (socket says 172.21.15.5:60064) 2024-03-18T15:34:03.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.572+0000 7f2aea59c640 1 -- 172.21.15.5:0/2767367979 learned_addr learned my addr 172.21.15.5:0/2767367979 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:03.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.572+0000 7f2ae9d9b640 1 -- 172.21.15.5:0/2767367979 >> v1:172.21.15.80:6789/0 conn(0x7f2aec0b26a0 legacy=0x7f2aec0b2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:03.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.572+0000 7f2ae9d9b640 1 -- 172.21.15.5:0/2767367979 >> v1:172.21.15.5:6789/0 conn(0x7f2aec0af790 legacy=0x7f2aec147780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:03.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.572+0000 7f2ae9d9b640 1 -- 172.21.15.5:0/2767367979 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aec0b4d00 con 0x7f2aec072b40 2024-03-18T15:34:03.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.572+0000 7f2ae959a640 1 -- 172.21.15.5:0/2767367979 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1684904246 0 0) 0x7f2aec0afb70 con 0x7f2aec0af790 2024-03-18T15:34:03.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.576+0000 7f2ae9d9b640 1 --2- 172.21.15.5:0/2767367979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2aec072b40 0x7f2aec06d070 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2ae00097c0 tx=0x7f2ae001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2fef93d37424fab server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:03.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.576+0000 7f2ae959a640 1 -- 172.21.15.5:0/2767367979 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae00b0070 con 0x7f2aec072b40 2024-03-18T15:34:03.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.576+0000 7f2ae959a640 1 -- 172.21.15.5:0/2767367979 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ae00ab600 con 0x7f2aec072b40 2024-03-18T15:34:03.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.576+0000 7f2ae959a640 1 -- 172.21.15.5:0/2767367979 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae00ab9d0 con 0x7f2aec072b40 2024-03-18T15:34:03.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.576+0000 7f2af08bf640 1 -- 172.21.15.5:0/2767367979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2aec072b40 msgr2=0x7f2aec06d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:03.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.576+0000 7f2af08bf640 1 --2- 172.21.15.5:0/2767367979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2aec072b40 0x7f2aec06d070 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2ae00097c0 tx=0x7f2ae001c3d0 comp rx=0 tx=0).stop 2024-03-18T15:34:03.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.576+0000 7f2af08bf640 1 -- 172.21.15.5:0/2767367979 shutdown_connections 2024-03-18T15:34:03.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.576+0000 7f2af08bf640 1 --2- 172.21.15.5:0/2767367979 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2aec072b40 0x7f2aec06d070 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:03.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.576+0000 7f2af08bf640 1 -- 172.21.15.5:0/2767367979 >> 172.21.15.5:0/2767367979 conn(0x7f2aec0b5670 msgr2=0x7f2aec0b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:03.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.576+0000 7f2af08bf640 1 -- 172.21.15.5:0/2767367979 shutdown_connections 2024-03-18T15:34:03.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.576+0000 7f2af08bf640 1 -- 172.21.15.5:0/2767367979 wait complete. 2024-03-18T15:34:03.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.576+0000 7f2af08bf640 1 Processor -- start 2024-03-18T15:34:03.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.576+0000 7f2af08bf640 1 -- start start 2024-03-18T15:34:03.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.576+0000 7f2af08bf640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2aec072b40 0x7f2aec13eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:03.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.576+0000 7f2af08bf640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2aec0af790 0x7f2aec140210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:03.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.576+0000 7f2af08bf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2aec0b26a0 0x7f2aec140cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:03.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.576+0000 7f2af08bf640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec0b41d0 con 0x7f2aec0b26a0 2024-03-18T15:34:03.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.576+0000 7f2af08bf640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec0b4050 con 0x7f2aec0af790 2024-03-18T15:34:03.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.576+0000 7f2af08bf640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f2aec0b4350 con 0x7f2aec072b40 2024-03-18T15:34:03.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.576+0000 7f2aead9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2aec0b26a0 0x7f2aec140cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:03.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.576+0000 7f2aead9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2aec0b26a0 0x7f2aec140cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50048/0 (socket says 172.21.15.5:50048) 2024-03-18T15:34:03.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.576+0000 7f2aead9d640 1 -- 172.21.15.5:0/2395036309 learned_addr learned my addr 172.21.15.5:0/2395036309 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:03.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.576+0000 7f2aea59c640 1 --2- 172.21.15.5:0/2395036309 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2aec072b40 0x7f2aec13eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:03.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.576+0000 7f2ae9d9b640 1 --2- 172.21.15.5:0/2395036309 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2aec0af790 0x7f2aec140210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:03.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.576+0000 7f2aead9d640 1 -- 172.21.15.5:0/2395036309 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2aec072b40 msgr2=0x7f2aec13eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:03.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.576+0000 7f2aead9d640 1 --2- 172.21.15.5:0/2395036309 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2aec072b40 0x7f2aec13eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:03.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.576+0000 7f2aead9d640 1 -- 172.21.15.5:0/2395036309 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2aec0af790 msgr2=0x7f2aec140210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:03.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.576+0000 7f2aead9d640 1 --2- 172.21.15.5:0/2395036309 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2aec0af790 0x7f2aec140210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:03.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.576+0000 7f2aead9d640 1 -- 172.21.15.5:0/2395036309 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aec141760 con 0x7f2aec0b26a0 2024-03-18T15:34:03.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.576+0000 7f2ae9d9b640 1 --2- 172.21.15.5:0/2395036309 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2aec0af790 0x7f2aec140210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:03.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.576+0000 7f2aead9d640 1 --2- 172.21.15.5:0/2395036309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2aec0b26a0 0x7f2aec140cc0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2ae400c780 tx=0x7f2ae400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:03.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.576+0000 7f2adaffd640 1 -- 172.21.15.5:0/2395036309 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4004b70 con 0x7f2aec0b26a0 2024-03-18T15:34:03.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.576+0000 7f2adaffd640 1 -- 172.21.15.5:0/2395036309 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ae4004d10 con 0x7f2aec0b26a0 2024-03-18T15:34:03.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.576+0000 7f2af08bf640 1 -- 172.21.15.5:0/2395036309 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2aec13f7a0 con 0x7f2aec0b26a0 2024-03-18T15:34:03.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.576+0000 7f2adaffd640 1 -- 172.21.15.5:0/2395036309 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae400aa30 con 0x7f2aec0b26a0 2024-03-18T15:34:03.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.576+0000 7f2af08bf640 1 -- 172.21.15.5:0/2395036309 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2aec14f7e0 con 0x7f2aec0b26a0 2024-03-18T15:34:03.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.580+0000 7f2af08bf640 1 -- 172.21.15.5:0/2395036309 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2aec04f550 con 0x7f2aec0b26a0 2024-03-18T15:34:03.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.580+0000 7f2adaffd640 1 -- 172.21.15.5:0/2395036309 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f2ae400abd0 con 0x7f2aec0b26a0 2024-03-18T15:34:03.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.580+0000 7f2adaffd640 1 --2- 172.21.15.5:0/2395036309 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f2acc067660 0x7f2acc069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:03.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.580+0000 7f2adaffd640 1 -- 172.21.15.5:0/2395036309 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2ae4085d70 con 0x7f2aec0b26a0 2024-03-18T15:34:03.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.580+0000 7f2aea59c640 1 --2- 172.21.15.5:0/2395036309 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f2acc067660 0x7f2acc069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:03.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.580+0000 7f2aea59c640 1 --2- 172.21.15.5:0/2395036309 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f2acc067660 0x7f2acc069b20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2adc000f70 tx=0x7f2adc007b60 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:03.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.584+0000 7f2adaffd640 1 -- 172.21.15.5:0/2395036309 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f2ae4056a80 con 0x7f2aec0b26a0 2024-03-18T15:34:03.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.740+0000 7f2af08bf640 1 -- 172.21.15.5:0/2395036309 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2aec0b77f0 con 0x7f2aec0b26a0 2024-03-18T15:34:03.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.740+0000 7f2adaffd640 1 -- 172.21.15.5:0/2395036309 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f2ae4056380 con 0x7f2aec0b26a0 2024-03-18T15:34:03.745 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-03-18T15:34:03.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.744+0000 7f2af08bf640 1 -- 172.21.15.5:0/2395036309 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f2acc067660 msgr2=0x7f2acc069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:03.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.744+0000 7f2af08bf640 1 --2- 172.21.15.5:0/2395036309 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f2acc067660 0x7f2acc069b20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2adc000f70 tx=0x7f2adc007b60 comp rx=0 tx=0).stop 2024-03-18T15:34:03.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.744+0000 7f2af08bf640 1 -- 172.21.15.5:0/2395036309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2aec0b26a0 msgr2=0x7f2aec140cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:03.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.744+0000 7f2af08bf640 1 --2- 172.21.15.5:0/2395036309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2aec0b26a0 0x7f2aec140cc0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2ae400c780 tx=0x7f2ae400cc40 comp rx=0 tx=0).stop 2024-03-18T15:34:03.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.744+0000 7f2af08bf640 1 -- 172.21.15.5:0/2395036309 shutdown_connections 2024-03-18T15:34:03.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.744+0000 7f2af08bf640 1 --2- 172.21.15.5:0/2395036309 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f2acc067660 0x7f2acc069b20 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:03.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.744+0000 7f2af08bf640 1 --2- 172.21.15.5:0/2395036309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2aec0b26a0 0x7f2aec140cc0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:03.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.744+0000 7f2af08bf640 1 --2- 172.21.15.5:0/2395036309 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2aec0af790 0x7f2aec140210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:03.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.744+0000 7f2af08bf640 1 --2- 172.21.15.5:0/2395036309 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2aec072b40 0x7f2aec13eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:03.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.744+0000 7f2af08bf640 1 -- 172.21.15.5:0/2395036309 >> 172.21.15.5:0/2395036309 conn(0x7f2aec0b5670 msgr2=0x7f2aec0aebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:03.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.744+0000 7f2af08bf640 1 -- 172.21.15.5:0/2395036309 shutdown_connections 2024-03-18T15:34:03.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.744+0000 7f2af08bf640 1 -- 172.21.15.5:0/2395036309 wait complete. 2024-03-18T15:34:03.764 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-18T15:34:03.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.888+0000 7fd59bcd4640 1 Processor -- start 2024-03-18T15:34:03.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.888+0000 7fd59bcd4640 1 -- start start 2024-03-18T15:34:03.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.892+0000 7fd59bcd4640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd59406d2d0 0x7fd5940b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:03.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.892+0000 7fd59bcd4640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd59413f6a0 0x7fd594145ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:03.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.892+0000 7fd59bcd4640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd59413fb10 con 0x7fd59413f260 2024-03-18T15:34:03.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.892+0000 7fd59bcd4640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fd5940728c0 con 0x7fd59406d2d0 2024-03-18T15:34:03.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.892+0000 7fd59bcd4640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fd5940b0ea0 con 0x7fd59413f6a0 2024-03-18T15:34:03.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.892+0000 7fd59926f640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd59413f260 0x7fd5941453a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60068/0 (socket says 172.21.15.5:60068) 2024-03-18T15:34:03.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.892+0000 7fd59926f640 1 -- 172.21.15.5:0/1108064510 learned_addr learned my addr 172.21.15.5:0/1108064510 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:03.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.892+0000 7fd599a70640 1 --2- 172.21.15.5:0/1108064510 >> v2:172.21.15.80:3300/0 conn(0x7fd59406d2d0 0x7fd5940b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:03.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.892+0000 7fd59a271640 1 --2- 172.21.15.5:0/1108064510 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd59413f6a0 0x7fd594145ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:03.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.892+0000 7fd598a6e640 1 -- 172.21.15.5:0/1108064510 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3118276837 0 0) 0x7fd59413fb10 con 0x7fd59413f260 2024-03-18T15:34:03.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.892+0000 7fd598a6e640 1 -- 172.21.15.5:0/1108064510 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd580003530 con 0x7fd59413f260 2024-03-18T15:34:03.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.892+0000 7fd59a271640 1 -- 172.21.15.5:0/1108064510 >> v2:172.21.15.80:3300/0 conn(0x7fd59406d2d0 msgr2=0x7fd5940b23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:03.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.892+0000 7fd59a271640 1 --2- 172.21.15.5:0/1108064510 >> v2:172.21.15.80:3300/0 conn(0x7fd59406d2d0 0x7fd5940b23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:03.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.892+0000 7fd59a271640 1 -- 172.21.15.5:0/1108064510 >> v1:172.21.15.5:6789/0 conn(0x7fd59413f260 legacy=0x7fd5941453a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:03.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.892+0000 7fd59a271640 1 -- 172.21.15.5:0/1108064510 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd59406d6b0 con 0x7fd59413f6a0 2024-03-18T15:34:03.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.892+0000 7fd599a70640 1 --2- 172.21.15.5:0/1108064510 >> v2:172.21.15.80:3300/0 conn(0x7fd59406d2d0 0x7fd5940b23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:34:03.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.896+0000 7fd59a271640 1 --2- 172.21.15.5:0/1108064510 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd59413f6a0 0x7fd594145ab0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd5900097c0 tx=0x7fd59001c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adfa25674b758611 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:03.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.896+0000 7fd598a6e640 1 -- 172.21.15.5:0/1108064510 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5900b1070 con 0x7fd59413f6a0 2024-03-18T15:34:03.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.896+0000 7fd598a6e640 1 -- 172.21.15.5:0/1108064510 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5900a8050 con 0x7fd59413f6a0 2024-03-18T15:34:03.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.896+0000 7fd598a6e640 1 -- 172.21.15.5:0/1108064510 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5900ac840 con 0x7fd59413f6a0 2024-03-18T15:34:03.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.896+0000 7fd59bcd4640 1 -- 172.21.15.5:0/1108064510 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd59413f6a0 msgr2=0x7fd594145ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:03.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.896+0000 7fd59bcd4640 1 --2- 172.21.15.5:0/1108064510 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd59413f6a0 0x7fd594145ab0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd5900097c0 tx=0x7fd59001c3e0 comp rx=0 tx=0).stop 2024-03-18T15:34:03.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.896+0000 7fd59bcd4640 1 -- 172.21.15.5:0/1108064510 shutdown_connections 2024-03-18T15:34:03.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.896+0000 7fd59bcd4640 1 --2- 172.21.15.5:0/1108064510 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd59413f6a0 0x7fd594145ab0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:03.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.896+0000 7fd59bcd4640 1 --2- 172.21.15.5:0/1108064510 >> v2:172.21.15.80:3300/0 conn(0x7fd59406d2d0 0x7fd5940b23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:03.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.896+0000 7fd59bcd4640 1 -- 172.21.15.5:0/1108064510 >> 172.21.15.5:0/1108064510 conn(0x7fd5940b5670 msgr2=0x7fd5940b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:03.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.896+0000 7fd59bcd4640 1 -- 172.21.15.5:0/1108064510 shutdown_connections 2024-03-18T15:34:03.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.896+0000 7fd59bcd4640 1 -- 172.21.15.5:0/1108064510 wait complete. 2024-03-18T15:34:03.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.896+0000 7fd59bcd4640 1 Processor -- start 2024-03-18T15:34:03.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.896+0000 7fd59bcd4640 1 -- start start 2024-03-18T15:34:03.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.896+0000 7fd59bcd4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd59406d2d0 0x7fd59413b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:03.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.896+0000 7fd59bcd4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd59413f260 0x7fd5940b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:03.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.896+0000 7fd59bcd4640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd59413f6a0 0x7fd5940b7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:03.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.896+0000 7fd59bcd4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd5940b0960 con 0x7fd59406d2d0 2024-03-18T15:34:03.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.896+0000 7fd59bcd4640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd5940b07e0 con 0x7fd59413f260 2024-03-18T15:34:03.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.896+0000 7fd59bcd4640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fd5940b5140 con 0x7fd59413f6a0 2024-03-18T15:34:03.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.896+0000 7fd599a70640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd59406d2d0 0x7fd59413b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:03.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.896+0000 7fd599a70640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd59406d2d0 0x7fd59413b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50050/0 (socket says 172.21.15.5:50050) 2024-03-18T15:34:03.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.896+0000 7fd599a70640 1 -- 172.21.15.5:0/3249050659 learned_addr learned my addr 172.21.15.5:0/3249050659 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:03.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.896+0000 7fd59a271640 1 --2- 172.21.15.5:0/3249050659 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd59413f6a0 0x7fd5940b7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:03.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.896+0000 7fd59926f640 1 --2- 172.21.15.5:0/3249050659 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd59413f260 0x7fd5940b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:03.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.896+0000 7fd599a70640 1 -- 172.21.15.5:0/3249050659 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd59413f6a0 msgr2=0x7fd5940b7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:03.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.896+0000 7fd599a70640 1 --2- 172.21.15.5:0/3249050659 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd59413f6a0 0x7fd5940b7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:03.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.896+0000 7fd599a70640 1 -- 172.21.15.5:0/3249050659 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd59413f260 msgr2=0x7fd5940b70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:03.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.896+0000 7fd599a70640 1 --2- 172.21.15.5:0/3249050659 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd59413f260 0x7fd5940b70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:03.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.896+0000 7fd599a70640 1 -- 172.21.15.5:0/3249050659 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5940adee0 con 0x7fd59406d2d0 2024-03-18T15:34:03.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.896+0000 7fd599a70640 1 --2- 172.21.15.5:0/3249050659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd59406d2d0 0x7fd59413b3d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd58c00efc0 tx=0x7fd58c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:03.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.896+0000 7fd58a7fc640 1 -- 172.21.15.5:0/3249050659 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58c018020 con 0x7fd59406d2d0 2024-03-18T15:34:03.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.896+0000 7fd58a7fc640 1 -- 172.21.15.5:0/3249050659 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd58c014680 con 0x7fd59406d2d0 2024-03-18T15:34:03.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.896+0000 7fd59bcd4640 1 -- 172.21.15.5:0/3249050659 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd59413abf0 con 0x7fd59406d2d0 2024-03-18T15:34:03.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.896+0000 7fd58a7fc640 1 -- 172.21.15.5:0/3249050659 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58c014a50 con 0x7fd59406d2d0 2024-03-18T15:34:03.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.896+0000 7fd59bcd4640 1 -- 172.21.15.5:0/3249050659 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd59413a910 con 0x7fd59406d2d0 2024-03-18T15:34:03.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.900+0000 7fd59bcd4640 1 -- 172.21.15.5:0/3249050659 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd59404f550 con 0x7fd59406d2d0 2024-03-18T15:34:03.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.900+0000 7fd58a7fc640 1 -- 172.21.15.5:0/3249050659 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fd58c01b020 con 0x7fd59406d2d0 2024-03-18T15:34:03.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.900+0000 7fd58a7fc640 1 --2- 172.21.15.5:0/3249050659 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd578067660 0x7fd578069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:03.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.900+0000 7fd59926f640 1 --2- 172.21.15.5:0/3249050659 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd578067660 0x7fd578069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:03.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.900+0000 7fd58a7fc640 1 -- 172.21.15.5:0/3249050659 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd58c086630 con 0x7fd59406d2d0 2024-03-18T15:34:03.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.900+0000 7fd59926f640 1 --2- 172.21.15.5:0/3249050659 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd578067660 0x7fd578069b20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd59413c140 tx=0x7fd584009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:03.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:03.904+0000 7fd58a7fc640 1 -- 172.21.15.5:0/3249050659 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fd58c022030 con 0x7fd59406d2d0 2024-03-18T15:34:04.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.072+0000 7fd59bcd4640 1 -- 172.21.15.5:0/3249050659 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd5940b1350 con 0x7fd59406d2d0 2024-03-18T15:34:04.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.072+0000 7fd58a7fc640 1 -- 172.21.15.5:0/3249050659 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7fd58c010040 con 0x7fd59406d2d0 2024-03-18T15:34:04.075 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-03-18T15:34:04.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.072+0000 7fd59bcd4640 1 -- 172.21.15.5:0/3249050659 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd578067660 msgr2=0x7fd578069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:04.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.072+0000 7fd59bcd4640 1 --2- 172.21.15.5:0/3249050659 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd578067660 0x7fd578069b20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd59413c140 tx=0x7fd584009040 comp rx=0 tx=0).stop 2024-03-18T15:34:04.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.072+0000 7fd59bcd4640 1 -- 172.21.15.5:0/3249050659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd59406d2d0 msgr2=0x7fd59413b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:04.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.072+0000 7fd59bcd4640 1 --2- 172.21.15.5:0/3249050659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd59406d2d0 0x7fd59413b3d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd58c00efc0 tx=0x7fd58c00c490 comp rx=0 tx=0).stop 2024-03-18T15:34:04.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.072+0000 7fd59bcd4640 1 -- 172.21.15.5:0/3249050659 shutdown_connections 2024-03-18T15:34:04.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.072+0000 7fd59bcd4640 1 --2- 172.21.15.5:0/3249050659 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd578067660 0x7fd578069b20 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:04.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.072+0000 7fd59bcd4640 1 --2- 172.21.15.5:0/3249050659 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd59413f6a0 0x7fd5940b7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:04.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.072+0000 7fd59bcd4640 1 --2- 172.21.15.5:0/3249050659 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd59413f260 0x7fd5940b70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:04.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.072+0000 7fd59bcd4640 1 --2- 172.21.15.5:0/3249050659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd59406d2d0 0x7fd59413b3d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:04.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.072+0000 7fd59bcd4640 1 -- 172.21.15.5:0/3249050659 >> 172.21.15.5:0/3249050659 conn(0x7fd5940b5670 msgr2=0x7fd59406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:04.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.076+0000 7fd59bcd4640 1 -- 172.21.15.5:0/3249050659 shutdown_connections 2024-03-18T15:34:04.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.076+0000 7fd59bcd4640 1 -- 172.21.15.5:0/3249050659 wait complete. 2024-03-18T15:34:04.095 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_cephfs_shell.TestCD.test_cd_with_args' 2024-03-18T15:34:04.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.216+0000 7f25906dc640 1 Processor -- start 2024-03-18T15:34:04.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.216+0000 7f25906dc640 1 -- start start 2024-03-18T15:34:04.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.216+0000 7f25906dc640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2588141a30 0x7f258814aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:04.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.216+0000 7f25906dc640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25880b5860 con 0x7f25880b78a0 2024-03-18T15:34:04.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.216+0000 7f25906dc640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f258806d390 con 0x7f2588072b20 2024-03-18T15:34:04.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.216+0000 7f25906dc640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f25880b7c80 con 0x7f2588141a30 2024-03-18T15:34:04.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.216+0000 7f258e478640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f25880b78a0 0x7f2588141320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60078/0 (socket says 172.21.15.5:60078) 2024-03-18T15:34:04.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.216+0000 7f258e478640 1 -- 172.21.15.5:0/2446025382 learned_addr learned my addr 172.21.15.5:0/2446025382 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:04.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.216+0000 7f258dc77640 1 --2- 172.21.15.5:0/2446025382 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2588141a30 0x7f258814aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:04.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.216+0000 7f258d476640 1 -- 172.21.15.5:0/2446025382 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3287196586 0 0) 0x7f25880b5860 con 0x7f25880b78a0 2024-03-18T15:34:04.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.216+0000 7f258d476640 1 -- 172.21.15.5:0/2446025382 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2574003530 con 0x7f25880b78a0 2024-03-18T15:34:04.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.216+0000 7f258d476640 1 -- 172.21.15.5:0/2446025382 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2154418970 0 0) 0x7f2574003530 con 0x7f25880b78a0 2024-03-18T15:34:04.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.216+0000 7f258d476640 1 -- 172.21.15.5:0/2446025382 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2588141a30 msgr2=0x7f258814aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:04.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.216+0000 7f258d476640 1 --2- 172.21.15.5:0/2446025382 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2588141a30 0x7f258814aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:04.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.216+0000 7f258d476640 1 -- 172.21.15.5:0/2446025382 >> v1:172.21.15.80:6789/0 conn(0x7f2588072b20 legacy=0x7f2588151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:04.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.216+0000 7f258d476640 1 -- 172.21.15.5:0/2446025382 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25880b5b40 con 0x7f25880b78a0 2024-03-18T15:34:04.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.216+0000 7f258d476640 1 -- 172.21.15.5:0/2446025382 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3837549003 0 0) 0x7f2584001680 con 0x7f25880b78a0 2024-03-18T15:34:04.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.216+0000 7f258d476640 1 -- 172.21.15.5:0/2446025382 >> v1:172.21.15.5:6789/0 conn(0x7f25880b78a0 legacy=0x7f2588141320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:04.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.216+0000 7f258d476640 1 --2- 172.21.15.5:0/2446025382 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2574003ad0 0x7f2574003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:04.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.216+0000 7f258d476640 1 --2- 172.21.15.5:0/2446025382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2574004810 0x7f2574004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:04.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.216+0000 7f258d476640 1 --2- 172.21.15.5:0/2446025382 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25740052f0 0x7f257400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:04.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.216+0000 7f258d476640 1 -- 172.21.15.5:0/2446025382 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f25880b7c80 con 0x7f2574004810 2024-03-18T15:34:04.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.216+0000 7f258d476640 1 -- 172.21.15.5:0/2446025382 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f257400c210 con 0x7f25740052f0 2024-03-18T15:34:04.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.216+0000 7f258d476640 1 -- 172.21.15.5:0/2446025382 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f257400c410 con 0x7f2574003ad0 2024-03-18T15:34:04.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.216+0000 7f258dc77640 1 --2- 172.21.15.5:0/2446025382 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2588141a30 0x7f258814aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:34:04.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.216+0000 7f258dc77640 1 --2- 172.21.15.5:0/2446025382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2574004810 0x7f2574004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:04.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.216+0000 7f258ec79640 1 --2- 172.21.15.5:0/2446025382 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25740052f0 0x7f257400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:04.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.216+0000 7f258e478640 1 --2- 172.21.15.5:0/2446025382 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2574003ad0 0x7f2574003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:04.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.216+0000 7f258dc77640 1 -- 172.21.15.5:0/2446025382 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2574003ad0 msgr2=0x7f2574003ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:04.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.216+0000 7f258dc77640 1 --2- 172.21.15.5:0/2446025382 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2574003ad0 0x7f2574003ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:04.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.216+0000 7f258dc77640 1 -- 172.21.15.5:0/2446025382 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25740052f0 msgr2=0x7f257400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:04.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.216+0000 7f258dc77640 1 --2- 172.21.15.5:0/2446025382 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25740052f0 0x7f257400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:04.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.216+0000 7f258dc77640 1 -- 172.21.15.5:0/2446025382 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2574003530 con 0x7f2574004810 2024-03-18T15:34:04.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.220+0000 7f258dc77640 1 --2- 172.21.15.5:0/2446025382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2574004810 0x7f2574004c90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f25880b2080 tx=0x7f257c01cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e043a31b7028406 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:04.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.220+0000 7f258d476640 1 -- 172.21.15.5:0/2446025382 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257c0ab070 con 0x7f2574004810 2024-03-18T15:34:04.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.220+0000 7f258d476640 1 -- 172.21.15.5:0/2446025382 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f257c0a9070 con 0x7f2574004810 2024-03-18T15:34:04.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.220+0000 7f258d476640 1 -- 172.21.15.5:0/2446025382 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257c009b50 con 0x7f2574004810 2024-03-18T15:34:04.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.220+0000 7f25906dc640 1 -- 172.21.15.5:0/2446025382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2574004810 msgr2=0x7f2574004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:04.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.220+0000 7f25906dc640 1 --2- 172.21.15.5:0/2446025382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2574004810 0x7f2574004c90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f25880b2080 tx=0x7f257c01cd10 comp rx=0 tx=0).stop 2024-03-18T15:34:04.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.220+0000 7f258ec79640 1 -- 172.21.15.5:0/2446025382 reap_dead start 2024-03-18T15:34:04.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.220+0000 7f25906dc640 1 -- 172.21.15.5:0/2446025382 shutdown_connections 2024-03-18T15:34:04.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.220+0000 7f25906dc640 1 -- 172.21.15.5:0/2446025382 >> 172.21.15.5:0/2446025382 conn(0x7f2588066fc0 msgr2=0x7f25880673d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:04.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.220+0000 7f25906dc640 1 -- 172.21.15.5:0/2446025382 shutdown_connections 2024-03-18T15:34:04.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.220+0000 7f25906dc640 1 -- 172.21.15.5:0/2446025382 wait complete. 2024-03-18T15:34:04.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.220+0000 7f25906dc640 1 Processor -- start 2024-03-18T15:34:04.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.220+0000 7f25906dc640 1 -- start start 2024-03-18T15:34:04.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.220+0000 7f25906dc640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2574004810 0x7f25881442b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:04.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.220+0000 7f25906dc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2588143390 0x7f25881421b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:04.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.220+0000 7f25906dc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25881426f0 0x7f2588137e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:04.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.220+0000 7f25906dc640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2588153ef0 con 0x7f2588143390 2024-03-18T15:34:04.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.220+0000 7f25906dc640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f258806dcc0 con 0x7f25881426f0 2024-03-18T15:34:04.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.220+0000 7f25906dc640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f2588154070 con 0x7f2574004810 2024-03-18T15:34:04.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.220+0000 7f258dc77640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2588143390 0x7f25881421b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:04.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.220+0000 7f258dc77640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2588143390 0x7f25881421b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50070/0 (socket says 172.21.15.5:50070) 2024-03-18T15:34:04.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.220+0000 7f258dc77640 1 -- 172.21.15.5:0/3905191835 learned_addr learned my addr 172.21.15.5:0/3905191835 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:04.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.220+0000 7f258ec79640 1 --2- 172.21.15.5:0/3905191835 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25881426f0 0x7f2588137e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:04.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.220+0000 7f258e478640 1 --2- 172.21.15.5:0/3905191835 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2574004810 0x7f25881442b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:04.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.220+0000 7f258dc77640 1 -- 172.21.15.5:0/3905191835 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2574004810 msgr2=0x7f25881442b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:04.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.220+0000 7f258dc77640 1 --2- 172.21.15.5:0/3905191835 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2574004810 0x7f25881442b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:04.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.220+0000 7f258dc77640 1 -- 172.21.15.5:0/3905191835 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25881426f0 msgr2=0x7f2588137e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:04.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.220+0000 7f258dc77640 1 --2- 172.21.15.5:0/3905191835 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25881426f0 0x7f2588137e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:04.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.220+0000 7f258dc77640 1 -- 172.21.15.5:0/3905191835 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2588143850 con 0x7f2588143390 2024-03-18T15:34:04.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.220+0000 7f258e478640 1 --2- 172.21.15.5:0/3905191835 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2574004810 0x7f25881442b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:04.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.220+0000 7f258dc77640 1 --2- 172.21.15.5:0/3905191835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2588143390 0x7f25881421b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f257c01c780 tx=0x7f2574005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:04.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.224+0000 7f257affd640 1 -- 172.21.15.5:0/3905191835 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257c0ab040 con 0x7f2588143390 2024-03-18T15:34:04.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.224+0000 7f257affd640 1 -- 172.21.15.5:0/3905191835 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f257c0b4070 con 0x7f2588143390 2024-03-18T15:34:04.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.224+0000 7f25906dc640 1 -- 172.21.15.5:0/3905191835 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2588143b30 con 0x7f2588143390 2024-03-18T15:34:04.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.224+0000 7f257affd640 1 -- 172.21.15.5:0/3905191835 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257c0a5d00 con 0x7f2588143390 2024-03-18T15:34:04.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.224+0000 7f25906dc640 1 -- 172.21.15.5:0/3905191835 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2588148d70 con 0x7f2588143390 2024-03-18T15:34:04.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.224+0000 7f25906dc640 1 -- 172.21.15.5:0/3905191835 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2588142ad0 con 0x7f2588143390 2024-03-18T15:34:04.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.224+0000 7f257affd640 1 -- 172.21.15.5:0/3905191835 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f257c004030 con 0x7f2588143390 2024-03-18T15:34:04.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.224+0000 7f257affd640 1 --2- 172.21.15.5:0/3905191835 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f256c067660 0x7f256c069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:04.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.224+0000 7f258e478640 1 --2- 172.21.15.5:0/3905191835 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f256c067660 0x7f256c069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:04.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.224+0000 7f257affd640 1 -- 172.21.15.5:0/3905191835 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f257c11f810 con 0x7f2588143390 2024-03-18T15:34:04.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.224+0000 7f258e478640 1 --2- 172.21.15.5:0/3905191835 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f256c067660 0x7f256c069b20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f258814a840 tx=0x7f2584001490 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:04.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.228+0000 7f257affd640 1 -- 172.21.15.5:0/3905191835 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f257c016a20 con 0x7f2588143390 2024-03-18T15:34:04.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:04.440+0000 7f25906dc640 1 -- 172.21.15.5:0/3905191835 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_cephfs_shell.TestCD.test_cd_with_args"]} v 0) -- 0x7f258804f550 con 0x7f2588143390 2024-03-18T15:34:05.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.084+0000 7f257affd640 1 -- 172.21.15.5:0/3905191835 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_cephfs_shell.TestCD.test_cd_with_args"]}]=0 v45) ==== 137+0+0 (secure 0 0 0) 0x7f257c0f0520 con 0x7f2588143390 2024-03-18T15:34:05.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.088+0000 7f25906dc640 1 -- 172.21.15.5:0/3905191835 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f256c067660 msgr2=0x7f256c069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:05.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.088+0000 7f25906dc640 1 --2- 172.21.15.5:0/3905191835 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f256c067660 0x7f256c069b20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f258814a840 tx=0x7f2584001490 comp rx=0 tx=0).stop 2024-03-18T15:34:05.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.088+0000 7f25906dc640 1 -- 172.21.15.5:0/3905191835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2588143390 msgr2=0x7f25881421b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:05.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.088+0000 7f25906dc640 1 --2- 172.21.15.5:0/3905191835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2588143390 0x7f25881421b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f257c01c780 tx=0x7f2574005840 comp rx=0 tx=0).stop 2024-03-18T15:34:05.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.088+0000 7f25906dc640 1 -- 172.21.15.5:0/3905191835 shutdown_connections 2024-03-18T15:34:05.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.088+0000 7f25906dc640 1 --2- 172.21.15.5:0/3905191835 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f256c067660 0x7f256c069b20 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:05.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.088+0000 7f25906dc640 1 --2- 172.21.15.5:0/3905191835 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25881426f0 0x7f2588137e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:05.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.088+0000 7f25906dc640 1 --2- 172.21.15.5:0/3905191835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2588143390 0x7f25881421b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:05.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.088+0000 7f25906dc640 1 --2- 172.21.15.5:0/3905191835 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2574004810 0x7f25881442b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:05.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.088+0000 7f25906dc640 1 -- 172.21.15.5:0/3905191835 >> 172.21.15.5:0/3905191835 conn(0x7f2588066fc0 msgr2=0x7f25880b69c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:05.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.088+0000 7f25906dc640 1 -- 172.21.15.5:0/3905191835 shutdown_connections 2024-03-18T15:34:05.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.088+0000 7f25906dc640 1 -- 172.21.15.5:0/3905191835 wait complete. 2024-03-18T15:34:05.110 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-18T15:34:05.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.240+0000 7f78b20f0640 1 Processor -- start 2024-03-18T15:34:05.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.240+0000 7f78b20f0640 1 -- start start 2024-03-18T15:34:05.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.240+0000 7f78b20f0640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f78ac14b030 0x7f78ac140ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:05.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.240+0000 7f78b20f0640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78ac141430 0x7f78ac151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:05.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.240+0000 7f78b20f0640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78ac0b9d50 con 0x7f78ac0b5710 2024-03-18T15:34:05.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.240+0000 7f78b20f0640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f78ac0b4320 con 0x7f78ac14b030 2024-03-18T15:34:05.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.240+0000 7f78b20f0640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f78ac0b4570 con 0x7f78ac141430 2024-03-18T15:34:05.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.240+0000 7f78aaffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f78ac14b030 0x7f78ac140ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:05.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.240+0000 7f78abfff640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78ac141430 0x7f78ac151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:05.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.240+0000 7f78ab7fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f78ac0b5710 0x7f78ac14a920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.5:52240/0 (socket says 172.21.15.5:52240) 2024-03-18T15:34:05.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.240+0000 7f78ab7fe640 1 -- 172.21.15.5:0/3898080860 learned_addr learned my addr 172.21.15.5:0/3898080860 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:05.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.240+0000 7f78aaffd640 1 --2- 172.21.15.5:0/3898080860 >> v2:172.21.15.80:3300/0 conn(0x7f78ac14b030 0x7f78ac140ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:34296/0 (socket says 172.21.15.5:34296) 2024-03-18T15:34:05.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.240+0000 7f78aaffd640 1 -- 172.21.15.5:0/3898080860 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78ac141430 msgr2=0x7f78ac151cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:05.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.240+0000 7f78aaffd640 1 --2- 172.21.15.5:0/3898080860 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78ac141430 0x7f78ac151cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:05.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.240+0000 7f78aaffd640 1 -- 172.21.15.5:0/3898080860 >> v1:172.21.15.80:6789/0 conn(0x7f78ac0b5710 legacy=0x7f78ac14a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:05.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.240+0000 7f78aaffd640 1 -- 172.21.15.5:0/3898080860 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78ac0b4c30 con 0x7f78ac14b030 2024-03-18T15:34:05.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.244+0000 7f78aaffd640 1 --2- 172.21.15.5:0/3898080860 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78ac14b030 0x7f78ac140ef0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f78a00097c0 tx=0x7f78a001c380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5733833b910cc1be server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:05.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.244+0000 7f78aa7fc640 1 -- 172.21.15.5:0/3898080860 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a00b0070 con 0x7f78ac14b030 2024-03-18T15:34:05.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.244+0000 7f78aa7fc640 1 -- 172.21.15.5:0/3898080860 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78a00ab600 con 0x7f78ac14b030 2024-03-18T15:34:05.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.244+0000 7f78aa7fc640 1 -- 172.21.15.5:0/3898080860 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a00ab9d0 con 0x7f78ac14b030 2024-03-18T15:34:05.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.244+0000 7f78b20f0640 1 -- 172.21.15.5:0/3898080860 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78ac14b030 msgr2=0x7f78ac140ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:05.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.244+0000 7f78b20f0640 1 --2- 172.21.15.5:0/3898080860 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78ac14b030 0x7f78ac140ef0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f78a00097c0 tx=0x7f78a001c380 comp rx=0 tx=0).stop 2024-03-18T15:34:05.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.244+0000 7f78b20f0640 1 -- 172.21.15.5:0/3898080860 shutdown_connections 2024-03-18T15:34:05.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.244+0000 7f78b20f0640 1 --2- 172.21.15.5:0/3898080860 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78ac141430 0x7f78ac151cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:05.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.244+0000 7f78b20f0640 1 --2- 172.21.15.5:0/3898080860 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78ac14b030 0x7f78ac140ef0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:05.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.244+0000 7f78b20f0640 1 -- 172.21.15.5:0/3898080860 >> 172.21.15.5:0/3898080860 conn(0x7f78ac0673c0 msgr2=0x7f78ac0677d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:05.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.244+0000 7f78b20f0640 1 -- 172.21.15.5:0/3898080860 shutdown_connections 2024-03-18T15:34:05.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.244+0000 7f78b20f0640 1 -- 172.21.15.5:0/3898080860 wait complete. 2024-03-18T15:34:05.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.244+0000 7f78b20f0640 1 Processor -- start 2024-03-18T15:34:05.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.244+0000 7f78b20f0640 1 -- start start 2024-03-18T15:34:05.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.248+0000 7f78b20f0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78ac0b5710 0x7f78ac142340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:05.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.248+0000 7f78b20f0640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78ac141430 0x7f78ac14a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:05.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.248+0000 7f78b20f0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78ac14b030 0x7f78ac13a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:05.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.248+0000 7f78b20f0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f78ac1419c0 con 0x7f78ac0b5710 2024-03-18T15:34:05.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.248+0000 7f78b20f0640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f78ac141840 con 0x7f78ac14b030 2024-03-18T15:34:05.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.248+0000 7f78b20f0640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f78ac154a10 con 0x7f78ac141430 2024-03-18T15:34:05.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.248+0000 7f78ab7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78ac0b5710 0x7f78ac142340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:05.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.248+0000 7f78ab7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78ac0b5710 0x7f78ac142340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50072/0 (socket says 172.21.15.5:50072) 2024-03-18T15:34:05.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.248+0000 7f78ab7fe640 1 -- 172.21.15.5:0/2448498410 learned_addr learned my addr 172.21.15.5:0/2448498410 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:05.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.248+0000 7f78abfff640 1 --2- 172.21.15.5:0/2448498410 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78ac14b030 0x7f78ac13a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:05.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.248+0000 7f78aaffd640 1 --2- 172.21.15.5:0/2448498410 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78ac141430 0x7f78ac14a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:05.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.248+0000 7f78ab7fe640 1 -- 172.21.15.5:0/2448498410 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78ac141430 msgr2=0x7f78ac14a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:05.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.248+0000 7f78ab7fe640 1 --2- 172.21.15.5:0/2448498410 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78ac141430 0x7f78ac14a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:05.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.248+0000 7f78ab7fe640 1 -- 172.21.15.5:0/2448498410 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78ac14b030 msgr2=0x7f78ac13a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:05.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.248+0000 7f78ab7fe640 1 --2- 172.21.15.5:0/2448498410 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78ac14b030 0x7f78ac13a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:05.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.248+0000 7f78ab7fe640 1 -- 172.21.15.5:0/2448498410 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78ac143510 con 0x7f78ac0b5710 2024-03-18T15:34:05.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.248+0000 7f78abfff640 1 --2- 172.21.15.5:0/2448498410 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78ac14b030 0x7f78ac13a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:34:05.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.248+0000 7f78ab7fe640 1 --2- 172.21.15.5:0/2448498410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78ac0b5710 0x7f78ac142340 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f78a400cb10 tx=0x7f78a400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:05.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.248+0000 7f7893fff640 1 -- 172.21.15.5:0/2448498410 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a4017070 con 0x7f78ac0b5710 2024-03-18T15:34:05.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.248+0000 7f7893fff640 1 -- 172.21.15.5:0/2448498410 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78a400ac70 con 0x7f78ac0b5710 2024-03-18T15:34:05.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.248+0000 7f78b20f0640 1 -- 172.21.15.5:0/2448498410 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78ac13a760 con 0x7f78ac0b5710 2024-03-18T15:34:05.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.248+0000 7f7893fff640 1 -- 172.21.15.5:0/2448498410 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a400ae60 con 0x7f78ac0b5710 2024-03-18T15:34:05.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.248+0000 7f78b20f0640 1 -- 172.21.15.5:0/2448498410 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78ac144390 con 0x7f78ac0b5710 2024-03-18T15:34:05.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.248+0000 7f78b20f0640 1 -- 172.21.15.5:0/2448498410 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78ac1437e0 con 0x7f78ac0b5710 2024-03-18T15:34:05.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.248+0000 7f7893fff640 1 -- 172.21.15.5:0/2448498410 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f78a401a070 con 0x7f78ac0b5710 2024-03-18T15:34:05.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.248+0000 7f7893fff640 1 --2- 172.21.15.5:0/2448498410 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f7898067660 0x7f7898069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:05.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.248+0000 7f7893fff640 1 -- 172.21.15.5:0/2448498410 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f78a4085770 con 0x7f78ac0b5710 2024-03-18T15:34:05.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.248+0000 7f78aaffd640 1 --2- 172.21.15.5:0/2448498410 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f7898067660 0x7f7898069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:05.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.248+0000 7f78aaffd640 1 --2- 172.21.15.5:0/2448498410 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f7898067660 0x7f7898069b20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f78a00ad000 tx=0x7f78a00a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:05.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.252+0000 7f7893fff640 1 -- 172.21.15.5:0/2448498410 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f78ac1437e0 con 0x7f78ac0b5710 2024-03-18T15:34:05.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.408+0000 7f78b20f0640 1 -- 172.21.15.5:0/2448498410 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f78ac04f550 con 0x7f78ac0b5710 2024-03-18T15:34:05.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.416+0000 7f7893fff640 1 -- 172.21.15.5:0/2448498410 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f78a40562d0 con 0x7f78ac0b5710 2024-03-18T15:34:05.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.416+0000 7f78b20f0640 1 -- 172.21.15.5:0/2448498410 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f7898067660 msgr2=0x7f7898069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:05.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.416+0000 7f78b20f0640 1 --2- 172.21.15.5:0/2448498410 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f7898067660 0x7f7898069b20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f78a00ad000 tx=0x7f78a00a6000 comp rx=0 tx=0).stop 2024-03-18T15:34:05.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.416+0000 7f78b20f0640 1 -- 172.21.15.5:0/2448498410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78ac0b5710 msgr2=0x7f78ac142340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:05.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.416+0000 7f78b20f0640 1 --2- 172.21.15.5:0/2448498410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78ac0b5710 0x7f78ac142340 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f78a400cb10 tx=0x7f78a400a060 comp rx=0 tx=0).stop 2024-03-18T15:34:05.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.416+0000 7f78b20f0640 1 -- 172.21.15.5:0/2448498410 shutdown_connections 2024-03-18T15:34:05.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.416+0000 7f78b20f0640 1 --2- 172.21.15.5:0/2448498410 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f7898067660 0x7f7898069b20 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:05.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.416+0000 7f78b20f0640 1 --2- 172.21.15.5:0/2448498410 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78ac14b030 0x7f78ac13a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:05.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.416+0000 7f78b20f0640 1 --2- 172.21.15.5:0/2448498410 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f78ac141430 0x7f78ac14a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:05.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.416+0000 7f78b20f0640 1 --2- 172.21.15.5:0/2448498410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78ac0b5710 0x7f78ac142340 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:05.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.416+0000 7f78b20f0640 1 -- 172.21.15.5:0/2448498410 >> 172.21.15.5:0/2448498410 conn(0x7f78ac0673c0 msgr2=0x7f78ac0b7de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:05.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.416+0000 7f78b20f0640 1 -- 172.21.15.5:0/2448498410 shutdown_connections 2024-03-18T15:34:05.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.416+0000 7f78b20f0640 1 -- 172.21.15.5:0/2448498410 wait complete. 2024-03-18T15:34:05.435 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-18T15:34:05.436 DEBUG:teuthology.orchestra.run.smithi005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-18T15:34:05.491 DEBUG:teuthology.orchestra.run.smithi080:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-18T15:34:05.540 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-18T15:34:05.540 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-03-18T15:34:05.540 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-18T15:34:05.545 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-03-18T15:34:05.546 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-18T15:34:05.594 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:34:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.760+0000 7ff3a2097640 1 Processor -- start 2024-03-18T15:34:05.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.760+0000 7ff3a2097640 1 -- start start 2024-03-18T15:34:05.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.760+0000 7ff3a2097640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff39c0b5710 0x7ff39c14a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:05.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.760+0000 7ff3a2097640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff39c141600 0x7ff39c151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:05.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.760+0000 7ff3a2097640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff39c141a50 con 0x7ff39c14ae60 2024-03-18T15:34:05.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.760+0000 7ff3a2097640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7ff39c0b4320 con 0x7ff39c0b5710 2024-03-18T15:34:05.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.760+0000 7ff3a2097640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7ff39c0b4570 con 0x7ff39c141600 2024-03-18T15:34:05.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.760+0000 7ff39b7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff39c0b5710 0x7ff39c14a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:05.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.760+0000 7ff39b7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff39c0b5710 0x7ff39c14a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50084/0 (socket says 172.21.15.5:50084) 2024-03-18T15:34:05.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.760+0000 7ff39affd640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff39c14ae60 0x7ff39c140ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60082/0 (socket says 172.21.15.5:60082) 2024-03-18T15:34:05.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.760+0000 7ff39b7fe640 1 -- 172.21.15.5:0/495479418 learned_addr learned my addr 172.21.15.5:0/495479418 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:05.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.760+0000 7ff39bfff640 1 --2- 172.21.15.5:0/495479418 >> v2:172.21.15.80:3300/0 conn(0x7ff39c141600 0x7ff39c151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:05.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.760+0000 7ff39b7fe640 1 -- 172.21.15.5:0/495479418 >> v2:172.21.15.80:3300/0 conn(0x7ff39c141600 msgr2=0x7ff39c151cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:05.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.760+0000 7ff39b7fe640 1 --2- 172.21.15.5:0/495479418 >> v2:172.21.15.80:3300/0 conn(0x7ff39c141600 0x7ff39c151cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:05.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.760+0000 7ff39b7fe640 1 -- 172.21.15.5:0/495479418 >> v1:172.21.15.5:6789/0 conn(0x7ff39c14ae60 legacy=0x7ff39c140ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:05.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.760+0000 7ff39b7fe640 1 -- 172.21.15.5:0/495479418 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff39c0b9d50 con 0x7ff39c0b5710 2024-03-18T15:34:05.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.760+0000 7ff39bfff640 1 --2- 172.21.15.5:0/495479418 >> v2:172.21.15.80:3300/0 conn(0x7ff39c141600 0x7ff39c151cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:05.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.764+0000 7ff39b7fe640 1 --2- 172.21.15.5:0/495479418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff39c0b5710 0x7ff39c14a920 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff39c0b6e10 tx=0x7ff38c01c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7961045dbb74fbd server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:05.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.764+0000 7ff39a7fc640 1 -- 172.21.15.5:0/495479418 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c0a5590 con 0x7ff39c0b5710 2024-03-18T15:34:05.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.764+0000 7ff39a7fc640 1 -- 172.21.15.5:0/495479418 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff38c0a5730 con 0x7ff39c0b5710 2024-03-18T15:34:05.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.764+0000 7ff39a7fc640 1 -- 172.21.15.5:0/495479418 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c0a5b00 con 0x7ff39c0b5710 2024-03-18T15:34:05.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.764+0000 7ff3a2097640 1 -- 172.21.15.5:0/495479418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff39c0b5710 msgr2=0x7ff39c14a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:05.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.764+0000 7ff3a2097640 1 --2- 172.21.15.5:0/495479418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff39c0b5710 0x7ff39c14a920 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff39c0b6e10 tx=0x7ff38c01c8d0 comp rx=0 tx=0).stop 2024-03-18T15:34:05.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.764+0000 7ff3a2097640 1 -- 172.21.15.5:0/495479418 shutdown_connections 2024-03-18T15:34:05.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.764+0000 7ff3a2097640 1 --2- 172.21.15.5:0/495479418 >> v2:172.21.15.80:3300/0 conn(0x7ff39c141600 0x7ff39c151cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:05.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.764+0000 7ff3a2097640 1 --2- 172.21.15.5:0/495479418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff39c0b5710 0x7ff39c14a920 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:05.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.764+0000 7ff3a2097640 1 -- 172.21.15.5:0/495479418 >> 172.21.15.5:0/495479418 conn(0x7ff39c0673c0 msgr2=0x7ff39c0677d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:05.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.764+0000 7ff3a2097640 1 -- 172.21.15.5:0/495479418 shutdown_connections 2024-03-18T15:34:05.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.764+0000 7ff3a2097640 1 -- 172.21.15.5:0/495479418 wait complete. 2024-03-18T15:34:05.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.764+0000 7ff3a2097640 1 Processor -- start 2024-03-18T15:34:05.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.764+0000 7ff3a2097640 1 -- start start 2024-03-18T15:34:05.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.768+0000 7ff3a2097640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff39c0b5710 0x7ff39c13ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:05.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.768+0000 7ff3a2097640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff39c141600 0x7ff39c143180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:05.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.768+0000 7ff3a2097640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff39c14ae60 0x7ff39c143cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:05.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.768+0000 7ff3a2097640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff39c0b50e0 con 0x7ff39c14ae60 2024-03-18T15:34:05.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.768+0000 7ff3a2097640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff39c0b5af0 con 0x7ff39c0b5710 2024-03-18T15:34:05.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.768+0000 7ff3a2097640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7ff39c0b5260 con 0x7ff39c141600 2024-03-18T15:34:05.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.768+0000 7ff39bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff39c14ae60 0x7ff39c143cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:05.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.768+0000 7ff39bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff39c14ae60 0x7ff39c143cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50096/0 (socket says 172.21.15.5:50096) 2024-03-18T15:34:05.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.768+0000 7ff39bfff640 1 -- 172.21.15.5:0/3271245156 learned_addr learned my addr 172.21.15.5:0/3271245156 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:05.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.768+0000 7ff39b7fe640 1 --2- 172.21.15.5:0/3271245156 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff39c0b5710 0x7ff39c13ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:05.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.768+0000 7ff39affd640 1 --2- 172.21.15.5:0/3271245156 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff39c141600 0x7ff39c143180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:05.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.768+0000 7ff39bfff640 1 -- 172.21.15.5:0/3271245156 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff39c141600 msgr2=0x7ff39c143180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:05.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.768+0000 7ff39bfff640 1 --2- 172.21.15.5:0/3271245156 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff39c141600 0x7ff39c143180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:05.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.768+0000 7ff39bfff640 1 -- 172.21.15.5:0/3271245156 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff39c0b5710 msgr2=0x7ff39c13ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:05.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.768+0000 7ff39bfff640 1 --2- 172.21.15.5:0/3271245156 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff39c0b5710 0x7ff39c13ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:05.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.768+0000 7ff39bfff640 1 -- 172.21.15.5:0/3271245156 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff39c1446b0 con 0x7ff39c14ae60 2024-03-18T15:34:05.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.768+0000 7ff39affd640 1 --2- 172.21.15.5:0/3271245156 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff39c141600 0x7ff39c143180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:05.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.768+0000 7ff39bfff640 1 --2- 172.21.15.5:0/3271245156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff39c14ae60 0x7ff39c143cc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff39400c820 tx=0x7ff39400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:05.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.768+0000 7ff383fff640 1 -- 172.21.15.5:0/3271245156 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff394015070 con 0x7ff39c14ae60 2024-03-18T15:34:05.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.768+0000 7ff383fff640 1 -- 172.21.15.5:0/3271245156 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff394007a70 con 0x7ff39c14ae60 2024-03-18T15:34:05.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.768+0000 7ff383fff640 1 -- 172.21.15.5:0/3271245156 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff394007e60 con 0x7ff39c14ae60 2024-03-18T15:34:05.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.768+0000 7ff3a2097640 1 -- 172.21.15.5:0/3271245156 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff39c142630 con 0x7ff39c14ae60 2024-03-18T15:34:05.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.768+0000 7ff3a2097640 1 -- 172.21.15.5:0/3271245156 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff39c142290 con 0x7ff39c14ae60 2024-03-18T15:34:05.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.768+0000 7ff3a2097640 1 -- 172.21.15.5:0/3271245156 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff39c04f550 con 0x7ff39c14ae60 2024-03-18T15:34:05.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.768+0000 7ff383fff640 1 -- 172.21.15.5:0/3271245156 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7ff394018070 con 0x7ff39c14ae60 2024-03-18T15:34:05.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.768+0000 7ff383fff640 1 --2- 172.21.15.5:0/3271245156 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7ff388067660 0x7ff388069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:05.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.768+0000 7ff383fff640 1 -- 172.21.15.5:0/3271245156 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff394085810 con 0x7ff39c14ae60 2024-03-18T15:34:05.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.768+0000 7ff39b7fe640 1 --2- 172.21.15.5:0/3271245156 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7ff388067660 0x7ff388069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:05.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.772+0000 7ff39b7fe640 1 --2- 172.21.15.5:0/3271245156 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7ff388067660 0x7ff388069b20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff38c0ae000 tx=0x7ff38c0a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:05.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.772+0000 7ff383fff640 1 -- 172.21.15.5:0/3271245156 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7ff394056480 con 0x7ff39c14ae60 2024-03-18T15:34:05.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.936+0000 7ff3a2097640 1 -- 172.21.15.5:0/3271245156 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff39c0b7140 con 0x7ff39c14ae60 2024-03-18T15:34:05.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.940+0000 7ff383fff640 1 -- 172.21.15.5:0/3271245156 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16449 (secure 0 0 0) 0x7ff394055d80 con 0x7ff39c14ae60 2024-03-18T15:34:05.943 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:34:05.943 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":21,"fsid":"ef54fe03-5aa7-4eb2-9b1e-66c9cc345b94","created":"2024-03-18T15:33:18.086614+0000","modified":"2024-03-18T15:33:43.984050+0000","last_up_change":"2024-03-18T15:33:29.229911+0000","last_in_change":"2024-03-18T15:33:20.323840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:28.963186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T15:33:40.533646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T15:33:41.323519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df1c754b-af59-4d16-8919-fbb6386b3c48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6804","nonce":976264754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6808","nonce":976264754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6815","nonce":976264754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6811","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6813","nonce":976264754}]},"public_addr":"172.21.15.5:6804/976264754","cluster_addr":"172.21.15.5:6808/976264754","heartbeat_back_addr":"172.21.15.5:6815/976264754","heartbeat_front_addr":"172.21.15.5:6813/976264754","state":["exists","up"]},{"osd":1,"uuid":"b9a08b5e-8af4-4144-bdba-d69ed1ac627a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6801","nonce":3901761126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6803","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6805","nonce":3901761126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6812","nonce":3901761126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6807","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6809","nonce":3901761126}]},"public_addr":"172.21.15.5:6801/3901761126","cluster_addr":"172.21.15.5:6805/3901761126","heartbeat_back_addr":"172.21.15.5:6812/3901761126","heartbeat_front_addr":"172.21.15.5:6809/3901761126","state":["exists","up"]},{"osd":2,"uuid":"c3e8bb76-e382-439d-b380-62ef92f1255a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6817","nonce":502968448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6819","nonce":502968448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6823","nonce":502968448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6821","nonce":502968448}]},"public_addr":"172.21.15.5:6817/502968448","cluster_addr":"172.21.15.5:6819/502968448","heartbeat_back_addr":"172.21.15.5:6823/502968448","heartbeat_front_addr":"172.21.15.5:6821/502968448","state":["exists","up"]},{"osd":3,"uuid":"e8c94625-f2cd-4eba-b53d-2e7356e1c02a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6825","nonce":2425932735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6827","nonce":2425932735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6831","nonce":2425932735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6829","nonce":2425932735}]},"public_addr":"172.21.15.5:6825/2425932735","cluster_addr":"172.21.15.5:6827/2425932735","heartbeat_back_addr":"172.21.15.5:6831/2425932735","heartbeat_front_addr":"172.21.15.5:6829/2425932735","state":["exists","up"]},{"osd":4,"uuid":"0099dd52-6265-4d32-82e4-3ef16e73add0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6817","nonce":976514285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6819","nonce":976514285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6823","nonce":976514285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6821","nonce":976514285}]},"public_addr":"172.21.15.80:6817/976514285","cluster_addr":"172.21.15.80:6819/976514285","heartbeat_back_addr":"172.21.15.80:6823/976514285","heartbeat_front_addr":"172.21.15.80:6821/976514285","state":["exists","up"]},{"osd":5,"uuid":"ed10c84f-8c39-41cc-9b64-7c482a94a48e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6825","nonce":627153447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6827","nonce":627153447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6831","nonce":627153447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6829","nonce":627153447}]},"public_addr":"172.21.15.80:6825/627153447","cluster_addr":"172.21.15.80:6827/627153447","heartbeat_back_addr":"172.21.15.80:6831/627153447","heartbeat_front_addr":"172.21.15.80:6829/627153447","state":["exists","up"]},{"osd":6,"uuid":"c7a3538a-7048-4fba-8781-b440f03c25db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6803","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6805","nonce":488342527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6807","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6809","nonce":488342527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6815","nonce":488342527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6811","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6813","nonce":488342527}]},"public_addr":"172.21.15.80:6805/488342527","cluster_addr":"172.21.15.80:6809/488342527","heartbeat_back_addr":"172.21.15.80:6815/488342527","heartbeat_front_addr":"172.21.15.80:6813/488342527","state":["exists","up"]},{"osd":7,"uuid":"f9418104-3a97-4110-bc3a-1f8151d0f757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6801","nonce":982831666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6804","nonce":982831666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6812","nonce":982831666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6808","nonce":982831666}]},"public_addr":"172.21.15.80:6801/982831666","cluster_addr":"172.21.15.80:6804/982831666","heartbeat_back_addr":"172.21.15.80:6812/982831666","heartbeat_front_addr":"172.21.15.80:6808/982831666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.970828+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.858116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.987556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.864603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.756293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.591502+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:34:05.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.940+0000 7ff3a2097640 1 -- 172.21.15.5:0/3271245156 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7ff388067660 msgr2=0x7ff388069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:05.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.940+0000 7ff3a2097640 1 --2- 172.21.15.5:0/3271245156 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7ff388067660 0x7ff388069b20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff38c0ae000 tx=0x7ff38c0a7070 comp rx=0 tx=0).stop 2024-03-18T15:34:05.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.940+0000 7ff3a2097640 1 -- 172.21.15.5:0/3271245156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff39c14ae60 msgr2=0x7ff39c143cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:05.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.940+0000 7ff3a2097640 1 --2- 172.21.15.5:0/3271245156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff39c14ae60 0x7ff39c143cc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff39400c820 tx=0x7ff39400cce0 comp rx=0 tx=0).stop 2024-03-18T15:34:05.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.940+0000 7ff3a2097640 1 -- 172.21.15.5:0/3271245156 shutdown_connections 2024-03-18T15:34:05.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.940+0000 7ff3a2097640 1 --2- 172.21.15.5:0/3271245156 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7ff388067660 0x7ff388069b20 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:05.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.940+0000 7ff3a2097640 1 --2- 172.21.15.5:0/3271245156 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff39c14ae60 0x7ff39c143cc0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:05.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.940+0000 7ff3a2097640 1 --2- 172.21.15.5:0/3271245156 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff39c141600 0x7ff39c143180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:05.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.940+0000 7ff3a2097640 1 --2- 172.21.15.5:0/3271245156 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff39c0b5710 0x7ff39c13ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:05.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.940+0000 7ff3a2097640 1 -- 172.21.15.5:0/3271245156 >> 172.21.15.5:0/3271245156 conn(0x7ff39c0673c0 msgr2=0x7ff39c148f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:05.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.944+0000 7ff3a2097640 1 -- 172.21.15.5:0/3271245156 shutdown_connections 2024-03-18T15:34:05.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:05.944+0000 7ff3a2097640 1 -- 172.21.15.5:0/3271245156 wait complete. 2024-03-18T15:34:05.962 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T15:33:28.963186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-18T15:33:40.533646+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-18T15:33:41.323519+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T15:34:05.962 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T15:34:06.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.084+0000 7fea8f71a640 1 Processor -- start 2024-03-18T15:34:06.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.084+0000 7fea8f71a640 1 -- start start 2024-03-18T15:34:06.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.084+0000 7fea8f71a640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fea8806d2d0 0x7fea880b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:06.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.084+0000 7fea8f71a640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fea8813f6a0 0x7fea88145ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:06.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.084+0000 7fea8f71a640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea880b0ea0 con 0x7fea8813f260 2024-03-18T15:34:06.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.084+0000 7fea8f71a640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fea880728c0 con 0x7fea8806d2d0 2024-03-18T15:34:06.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.084+0000 7fea8f71a640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fea8806d6b0 con 0x7fea8813f6a0 2024-03-18T15:34:06.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.084+0000 7fea8d4b6640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fea8806d2d0 0x7fea880b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:06.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.084+0000 7fea8d4b6640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fea8806d2d0 0x7fea880b23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50104/0 (socket says 172.21.15.5:50104) 2024-03-18T15:34:06.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.084+0000 7fea8d4b6640 1 -- 172.21.15.5:0/2466607508 learned_addr learned my addr 172.21.15.5:0/2466607508 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:06.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.084+0000 7fea8ccb5640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fea8813f260 0x7fea881453a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60098/0 (socket says 172.21.15.5:60098) 2024-03-18T15:34:06.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.084+0000 7fea8dcb7640 1 --2- 172.21.15.5:0/2466607508 >> v2:172.21.15.80:3300/0 conn(0x7fea8813f6a0 0x7fea88145ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:06.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.084+0000 7fea8d4b6640 1 -- 172.21.15.5:0/2466607508 >> v2:172.21.15.80:3300/0 conn(0x7fea8813f6a0 msgr2=0x7fea88145ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:06.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.084+0000 7fea8d4b6640 1 --2- 172.21.15.5:0/2466607508 >> v2:172.21.15.80:3300/0 conn(0x7fea8813f6a0 0x7fea88145ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:06.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.084+0000 7fea8d4b6640 1 -- 172.21.15.5:0/2466607508 >> v1:172.21.15.5:6789/0 conn(0x7fea8813f260 legacy=0x7fea881453a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:06.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.084+0000 7fea7ffff640 1 -- 172.21.15.5:0/2466607508 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 252371141 0 0) 0x7fea880b0ea0 con 0x7fea8813f260 2024-03-18T15:34:06.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.084+0000 7fea8d4b6640 1 -- 172.21.15.5:0/2466607508 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea8813faf0 con 0x7fea8806d2d0 2024-03-18T15:34:06.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.084+0000 7fea8dcb7640 1 --2- 172.21.15.5:0/2466607508 >> v2:172.21.15.80:3300/0 conn(0x7fea8813f6a0 0x7fea88145ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:34:06.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.088+0000 7fea8d4b6640 1 --2- 172.21.15.5:0/2466607508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea8806d2d0 0x7fea880b23a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fea8806f230 tx=0x7fea8401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba0102f7c8d28fac server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:06.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.088+0000 7fea7ffff640 1 -- 172.21.15.5:0/2466607508 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea840ab660 con 0x7fea8806d2d0 2024-03-18T15:34:06.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.088+0000 7fea7ffff640 1 -- 172.21.15.5:0/2466607508 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea840b0070 con 0x7fea8806d2d0 2024-03-18T15:34:06.091 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.088+0000 7fea7ffff640 1 -- 172.21.15.5:0/2466607508 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea840aba20 con 0x7fea8806d2d0 2024-03-18T15:34:06.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.088+0000 7fea8f71a640 1 -- 172.21.15.5:0/2466607508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea8806d2d0 msgr2=0x7fea880b23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:06.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.088+0000 7fea8f71a640 1 --2- 172.21.15.5:0/2466607508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea8806d2d0 0x7fea880b23a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fea8806f230 tx=0x7fea8401c460 comp rx=0 tx=0).stop 2024-03-18T15:34:06.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.088+0000 7fea8f71a640 1 -- 172.21.15.5:0/2466607508 shutdown_connections 2024-03-18T15:34:06.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.088+0000 7fea8f71a640 1 --2- 172.21.15.5:0/2466607508 >> v2:172.21.15.80:3300/0 conn(0x7fea8813f6a0 0x7fea88145ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:06.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.088+0000 7fea8f71a640 1 --2- 172.21.15.5:0/2466607508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea8806d2d0 0x7fea880b23a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:06.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.088+0000 7fea8f71a640 1 -- 172.21.15.5:0/2466607508 >> 172.21.15.5:0/2466607508 conn(0x7fea880b5670 msgr2=0x7fea880b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:06.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.088+0000 7fea8f71a640 1 -- 172.21.15.5:0/2466607508 shutdown_connections 2024-03-18T15:34:06.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.088+0000 7fea8f71a640 1 -- 172.21.15.5:0/2466607508 wait complete. 2024-03-18T15:34:06.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.088+0000 7fea8f71a640 1 Processor -- start 2024-03-18T15:34:06.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.088+0000 7fea8f71a640 1 -- start start 2024-03-18T15:34:06.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.092+0000 7fea8f71a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea8806d2d0 0x7fea880af5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:06.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.092+0000 7fea8f71a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea8813f260 0x7fea881e2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:06.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.092+0000 7fea8f71a640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fea8813f6a0 0x7fea880b7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:06.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.092+0000 7fea8f71a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fea880b0960 con 0x7fea8806d2d0 2024-03-18T15:34:06.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.092+0000 7fea8f71a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fea880b07e0 con 0x7fea8813f260 2024-03-18T15:34:06.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.092+0000 7fea8f71a640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fea88072d50 con 0x7fea8813f6a0 2024-03-18T15:34:06.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.092+0000 7fea8d4b6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea8806d2d0 0x7fea880af5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:06.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.092+0000 7fea8d4b6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea8806d2d0 0x7fea880af5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50110/0 (socket says 172.21.15.5:50110) 2024-03-18T15:34:06.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.092+0000 7fea8d4b6640 1 -- 172.21.15.5:0/1693520027 learned_addr learned my addr 172.21.15.5:0/1693520027 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:06.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.092+0000 7fea8dcb7640 1 --2- 172.21.15.5:0/1693520027 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fea8813f6a0 0x7fea880b7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:06.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.092+0000 7fea8ccb5640 1 --2- 172.21.15.5:0/1693520027 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea8813f260 0x7fea881e2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:06.095 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.092+0000 7fea8d4b6640 1 -- 172.21.15.5:0/1693520027 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fea8813f6a0 msgr2=0x7fea880b7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:06.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.092+0000 7fea8d4b6640 1 --2- 172.21.15.5:0/1693520027 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fea8813f6a0 0x7fea880b7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:06.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.092+0000 7fea8d4b6640 1 -- 172.21.15.5:0/1693520027 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea8813f260 msgr2=0x7fea881e2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:06.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.092+0000 7fea8d4b6640 1 --2- 172.21.15.5:0/1693520027 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea8813f260 0x7fea881e2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:06.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.092+0000 7fea8d4b6640 1 -- 172.21.15.5:0/1693520027 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea880b8170 con 0x7fea8806d2d0 2024-03-18T15:34:06.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.092+0000 7fea8d4b6640 1 --2- 172.21.15.5:0/1693520027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea8806d2d0 0x7fea880af5d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fea84019070 tx=0x7fea8401aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:06.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.092+0000 7fea7dffb640 1 -- 172.21.15.5:0/1693520027 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea840be5f0 con 0x7fea8806d2d0 2024-03-18T15:34:06.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.092+0000 7fea7dffb640 1 -- 172.21.15.5:0/1693520027 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea840be790 con 0x7fea8806d2d0 2024-03-18T15:34:06.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.092+0000 7fea8f71a640 1 -- 172.21.15.5:0/1693520027 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea880b8410 con 0x7fea8806d2d0 2024-03-18T15:34:06.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.092+0000 7fea7dffb640 1 -- 172.21.15.5:0/1693520027 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea840beb60 con 0x7fea8806d2d0 2024-03-18T15:34:06.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.092+0000 7fea8f71a640 1 -- 172.21.15.5:0/1693520027 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea88137e70 con 0x7fea8806d2d0 2024-03-18T15:34:06.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.092+0000 7fea8f71a640 1 -- 172.21.15.5:0/1693520027 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea8804f550 con 0x7fea8806d2d0 2024-03-18T15:34:06.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.092+0000 7fea7dffb640 1 -- 172.21.15.5:0/1693520027 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fea840b43f0 con 0x7fea8806d2d0 2024-03-18T15:34:06.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.092+0000 7fea7dffb640 1 --2- 172.21.15.5:0/1693520027 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fea6c067660 0x7fea6c069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:06.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.092+0000 7fea7dffb640 1 -- 172.21.15.5:0/1693520027 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fea84120140 con 0x7fea8806d2d0 2024-03-18T15:34:06.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.092+0000 7fea8ccb5640 1 --2- 172.21.15.5:0/1693520027 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fea6c067660 0x7fea6c069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:06.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.092+0000 7fea8ccb5640 1 --2- 172.21.15.5:0/1693520027 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fea6c067660 0x7fea6c069b20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fea8813ba20 tx=0x7fea78007b60 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:06.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.096+0000 7fea7dffb640 1 -- 172.21.15.5:0/1693520027 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fea840c4060 con 0x7fea8806d2d0 2024-03-18T15:34:06.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.252+0000 7fea8f71a640 1 -- 172.21.15.5:0/1693520027 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fea880adab0 con 0x7fea8806d2d0 2024-03-18T15:34:06.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.252+0000 7fea7dffb640 1 -- 172.21.15.5:0/1693520027 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7fea84016110 con 0x7fea8806d2d0 2024-03-18T15:34:06.255 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-03-18T15:34:06.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.252+0000 7fea8f71a640 1 -- 172.21.15.5:0/1693520027 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fea6c067660 msgr2=0x7fea6c069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:06.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.252+0000 7fea8f71a640 1 --2- 172.21.15.5:0/1693520027 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fea6c067660 0x7fea6c069b20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fea8813ba20 tx=0x7fea78007b60 comp rx=0 tx=0).stop 2024-03-18T15:34:06.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.252+0000 7fea8f71a640 1 -- 172.21.15.5:0/1693520027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea8806d2d0 msgr2=0x7fea880af5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:06.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.252+0000 7fea8f71a640 1 --2- 172.21.15.5:0/1693520027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea8806d2d0 0x7fea880af5d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fea84019070 tx=0x7fea8401aca0 comp rx=0 tx=0).stop 2024-03-18T15:34:06.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.252+0000 7fea8f71a640 1 -- 172.21.15.5:0/1693520027 shutdown_connections 2024-03-18T15:34:06.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.256+0000 7fea8f71a640 1 --2- 172.21.15.5:0/1693520027 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fea6c067660 0x7fea6c069b20 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:06.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.256+0000 7fea8f71a640 1 --2- 172.21.15.5:0/1693520027 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fea8813f6a0 0x7fea880b7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:06.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.256+0000 7fea8f71a640 1 --2- 172.21.15.5:0/1693520027 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea8813f260 0x7fea881e2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:06.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.256+0000 7fea8f71a640 1 --2- 172.21.15.5:0/1693520027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fea8806d2d0 0x7fea880af5d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:06.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.256+0000 7fea8f71a640 1 -- 172.21.15.5:0/1693520027 >> 172.21.15.5:0/1693520027 conn(0x7fea880b5670 msgr2=0x7fea8806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:06.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.256+0000 7fea8f71a640 1 -- 172.21.15.5:0/1693520027 shutdown_connections 2024-03-18T15:34:06.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.256+0000 7fea8f71a640 1 -- 172.21.15.5:0/1693520027 wait complete. 2024-03-18T15:34:06.273 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-18T15:34:06.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.384+0000 7fc04931f640 1 Processor -- start 2024-03-18T15:34:06.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.384+0000 7fc04931f640 1 -- start start 2024-03-18T15:34:06.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.384+0000 7fc04931f640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc04406d2d0 0x7fc0440b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:06.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.384+0000 7fc04931f640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc04413f6a0 0x7fc044145ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.384+0000 7fc04931f640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0440b0ea0 con 0x7fc04413f260 2024-03-18T15:34:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.384+0000 7fc04931f640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fc0440728c0 con 0x7fc04406d2d0 2024-03-18T15:34:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.384+0000 7fc04931f640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fc04406d6b0 con 0x7fc04413f6a0 2024-03-18T15:34:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.384+0000 7fc042ffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc04406d2d0 0x7fc0440b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.384+0000 7fc0427fc640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fc04413f260 0x7fc0441453a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60100/0 (socket says 172.21.15.5:60100) 2024-03-18T15:34:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.384+0000 7fc0427fc640 1 -- 172.21.15.5:0/4025763102 learned_addr learned my addr 172.21.15.5:0/4025763102 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.384+0000 7fc0437fe640 1 --2- 172.21.15.5:0/4025763102 >> v2:172.21.15.80:3300/0 conn(0x7fc04413f6a0 0x7fc044145ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.384+0000 7fc041ffb640 1 -- 172.21.15.5:0/4025763102 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 821465874 0 0) 0x7fc0440b0ea0 con 0x7fc04413f260 2024-03-18T15:34:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.384+0000 7fc041ffb640 1 -- 172.21.15.5:0/4025763102 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc02c003530 con 0x7fc04413f260 2024-03-18T15:34:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.384+0000 7fc042ffd640 1 -- 172.21.15.5:0/4025763102 >> v2:172.21.15.80:3300/0 conn(0x7fc04413f6a0 msgr2=0x7fc044145ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.384+0000 7fc042ffd640 1 --2- 172.21.15.5:0/4025763102 >> v2:172.21.15.80:3300/0 conn(0x7fc04413f6a0 0x7fc044145ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.384+0000 7fc042ffd640 1 -- 172.21.15.5:0/4025763102 >> v1:172.21.15.5:6789/0 conn(0x7fc04413f260 legacy=0x7fc0441453a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:06.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.384+0000 7fc042ffd640 1 -- 172.21.15.5:0/4025763102 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc04413faf0 con 0x7fc04406d2d0 2024-03-18T15:34:06.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.388+0000 7fc042ffd640 1 --2- 172.21.15.5:0/4025763102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc04406d2d0 0x7fc0440b23a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc04406f230 tx=0x7fc03401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccb43895e5107fd3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:06.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.388+0000 7fc041ffb640 1 -- 172.21.15.5:0/4025763102 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0340b1070 con 0x7fc04406d2d0 2024-03-18T15:34:06.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.388+0000 7fc041ffb640 1 -- 172.21.15.5:0/4025763102 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0340ac620 con 0x7fc04406d2d0 2024-03-18T15:34:06.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.388+0000 7fc041ffb640 1 -- 172.21.15.5:0/4025763102 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0340ac9f0 con 0x7fc04406d2d0 2024-03-18T15:34:06.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.388+0000 7fc04931f640 1 -- 172.21.15.5:0/4025763102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc04406d2d0 msgr2=0x7fc0440b23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:06.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.388+0000 7fc04931f640 1 --2- 172.21.15.5:0/4025763102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc04406d2d0 0x7fc0440b23a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc04406f230 tx=0x7fc03401c3c0 comp rx=0 tx=0).stop 2024-03-18T15:34:06.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.388+0000 7fc04931f640 1 -- 172.21.15.5:0/4025763102 shutdown_connections 2024-03-18T15:34:06.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.388+0000 7fc04931f640 1 --2- 172.21.15.5:0/4025763102 >> v2:172.21.15.80:3300/0 conn(0x7fc04413f6a0 0x7fc044145ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:06.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.388+0000 7fc04931f640 1 --2- 172.21.15.5:0/4025763102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc04406d2d0 0x7fc0440b23a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:06.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.388+0000 7fc04931f640 1 -- 172.21.15.5:0/4025763102 >> 172.21.15.5:0/4025763102 conn(0x7fc0440b5670 msgr2=0x7fc0440b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:06.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.388+0000 7fc04931f640 1 -- 172.21.15.5:0/4025763102 shutdown_connections 2024-03-18T15:34:06.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.388+0000 7fc04931f640 1 -- 172.21.15.5:0/4025763102 wait complete. 2024-03-18T15:34:06.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.388+0000 7fc04931f640 1 Processor -- start 2024-03-18T15:34:06.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.388+0000 7fc04931f640 1 -- start start 2024-03-18T15:34:06.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.388+0000 7fc04931f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc04406d2d0 0x7fc044139810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:06.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.388+0000 7fc04931f640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc04413f260 0x7fc0440b7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:06.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.388+0000 7fc04931f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc04413f6a0 0x7fc04413b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:06.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.388+0000 7fc04931f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc0440b0960 con 0x7fc04413f6a0 2024-03-18T15:34:06.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.388+0000 7fc04931f640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc0440b07e0 con 0x7fc04406d2d0 2024-03-18T15:34:06.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.388+0000 7fc04931f640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fc044072d50 con 0x7fc04413f260 2024-03-18T15:34:06.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.388+0000 7fc0437fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc04413f6a0 0x7fc04413b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:06.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.388+0000 7fc0437fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc04413f6a0 0x7fc04413b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50122/0 (socket says 172.21.15.5:50122) 2024-03-18T15:34:06.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.388+0000 7fc0437fe640 1 -- 172.21.15.5:0/930948294 learned_addr learned my addr 172.21.15.5:0/930948294 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:06.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.388+0000 7fc042ffd640 1 --2- 172.21.15.5:0/930948294 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc04406d2d0 0x7fc044139810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:06.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.388+0000 7fc0427fc640 1 --2- 172.21.15.5:0/930948294 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc04413f260 0x7fc0440b7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:06.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.388+0000 7fc0437fe640 1 -- 172.21.15.5:0/930948294 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc04413f260 msgr2=0x7fc0440b7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:06.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.388+0000 7fc0437fe640 1 --2- 172.21.15.5:0/930948294 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc04413f260 0x7fc0440b7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:06.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.388+0000 7fc0437fe640 1 -- 172.21.15.5:0/930948294 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc04406d2d0 msgr2=0x7fc044139810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:06.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.388+0000 7fc0437fe640 1 --2- 172.21.15.5:0/930948294 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc04406d2d0 0x7fc044139810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:06.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.388+0000 7fc0437fe640 1 -- 172.21.15.5:0/930948294 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc04413ae30 con 0x7fc04413f6a0 2024-03-18T15:34:06.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.388+0000 7fc042ffd640 1 --2- 172.21.15.5:0/930948294 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc04406d2d0 0x7fc044139810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:06.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.388+0000 7fc0437fe640 1 --2- 172.21.15.5:0/930948294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc04413f6a0 0x7fc04413b250 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc03c007e90 tx=0x7fc03c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:06.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.388+0000 7fc02b7fe640 1 -- 172.21.15.5:0/930948294 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03c014420 con 0x7fc04413f6a0 2024-03-18T15:34:06.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.388+0000 7fc02b7fe640 1 -- 172.21.15.5:0/930948294 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc03c019070 con 0x7fc04413f6a0 2024-03-18T15:34:06.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.388+0000 7fc04931f640 1 -- 172.21.15.5:0/930948294 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0441ddb00 con 0x7fc04413f6a0 2024-03-18T15:34:06.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.388+0000 7fc02b7fe640 1 -- 172.21.15.5:0/930948294 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc03c014800 con 0x7fc04413f6a0 2024-03-18T15:34:06.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.388+0000 7fc04931f640 1 -- 172.21.15.5:0/930948294 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0441de0c0 con 0x7fc04413f6a0 2024-03-18T15:34:06.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.392+0000 7fc04931f640 1 -- 172.21.15.5:0/930948294 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0441dd8f0 con 0x7fc04413f6a0 2024-03-18T15:34:06.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.392+0000 7fc02b7fe640 1 -- 172.21.15.5:0/930948294 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fc03c0149a0 con 0x7fc04413f6a0 2024-03-18T15:34:06.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.392+0000 7fc02b7fe640 1 --2- 172.21.15.5:0/930948294 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fc030067690 0x7fc030069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:06.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.392+0000 7fc02b7fe640 1 -- 172.21.15.5:0/930948294 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc03c086060 con 0x7fc04413f6a0 2024-03-18T15:34:06.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.392+0000 7fc042ffd640 1 --2- 172.21.15.5:0/930948294 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fc030067690 0x7fc030069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:06.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.392+0000 7fc042ffd640 1 --2- 172.21.15.5:0/930948294 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fc030067690 0x7fc030069b50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc0340ae000 tx=0x7fc034008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:06.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.396+0000 7fc02b7fe640 1 -- 172.21.15.5:0/930948294 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fc03c056d70 con 0x7fc04413f6a0 2024-03-18T15:34:06.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.552+0000 7fc04931f640 1 -- 172.21.15.5:0/930948294 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc0441de290 con 0x7fc04413f6a0 2024-03-18T15:34:06.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.552+0000 7fc02b7fe640 1 -- 172.21.15.5:0/930948294 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7fc03c056670 con 0x7fc04413f6a0 2024-03-18T15:34:06.556 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-03-18T15:34:06.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.556+0000 7fc04931f640 1 -- 172.21.15.5:0/930948294 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fc030067690 msgr2=0x7fc030069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:06.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.556+0000 7fc04931f640 1 --2- 172.21.15.5:0/930948294 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fc030067690 0x7fc030069b50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc0340ae000 tx=0x7fc034008040 comp rx=0 tx=0).stop 2024-03-18T15:34:06.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.556+0000 7fc04931f640 1 -- 172.21.15.5:0/930948294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc04413f6a0 msgr2=0x7fc04413b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:06.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.556+0000 7fc04931f640 1 --2- 172.21.15.5:0/930948294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc04413f6a0 0x7fc04413b250 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc03c007e90 tx=0x7fc03c00e4b0 comp rx=0 tx=0).stop 2024-03-18T15:34:06.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.556+0000 7fc04931f640 1 -- 172.21.15.5:0/930948294 shutdown_connections 2024-03-18T15:34:06.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.556+0000 7fc04931f640 1 --2- 172.21.15.5:0/930948294 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fc030067690 0x7fc030069b50 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:06.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.556+0000 7fc04931f640 1 --2- 172.21.15.5:0/930948294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc04413f6a0 0x7fc04413b250 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:06.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.556+0000 7fc04931f640 1 --2- 172.21.15.5:0/930948294 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc04413f260 0x7fc0440b7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:06.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.556+0000 7fc04931f640 1 --2- 172.21.15.5:0/930948294 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc04406d2d0 0x7fc044139810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:06.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.556+0000 7fc04931f640 1 -- 172.21.15.5:0/930948294 >> 172.21.15.5:0/930948294 conn(0x7fc0440b5670 msgr2=0x7fc04413cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:06.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.556+0000 7fc04931f640 1 -- 172.21.15.5:0/930948294 shutdown_connections 2024-03-18T15:34:06.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.556+0000 7fc04931f640 1 -- 172.21.15.5:0/930948294 wait complete. 2024-03-18T15:34:06.574 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-18T15:34:06.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.680+0000 7fbf5d63c640 1 Processor -- start 2024-03-18T15:34:06.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.684+0000 7fbf5d63c640 1 -- start start 2024-03-18T15:34:06.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.684+0000 7fbf5d63c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fbf580b5710 0x7fbf5814a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:06.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.684+0000 7fbf5d63c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbf5814ae60 0x7fbf58140ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:06.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.684+0000 7fbf5d63c640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf580b9d50 con 0x7fbf58141430 2024-03-18T15:34:06.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.684+0000 7fbf5d63c640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fbf580b4320 con 0x7fbf5814ae60 2024-03-18T15:34:06.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.684+0000 7fbf5d63c640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fbf580b4570 con 0x7fbf580b5710 2024-03-18T15:34:06.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.684+0000 7fbf567fc640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbf5814ae60 0x7fbf58140ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:06.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.684+0000 7fbf577fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fbf58141430 0x7fbf580b7690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60114/0 (socket says 172.21.15.5:60114) 2024-03-18T15:34:06.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.684+0000 7fbf577fe640 1 -- 172.21.15.5:0/2203710115 learned_addr learned my addr 172.21.15.5:0/2203710115 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:06.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.684+0000 7fbf56ffd640 1 --2- 172.21.15.5:0/2203710115 >> v2:172.21.15.80:3300/0 conn(0x7fbf580b5710 0x7fbf5814a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:06.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.684+0000 7fbf567fc640 1 -- 172.21.15.5:0/2203710115 >> v2:172.21.15.80:3300/0 conn(0x7fbf580b5710 msgr2=0x7fbf5814a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:06.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.684+0000 7fbf567fc640 1 --2- 172.21.15.5:0/2203710115 >> v2:172.21.15.80:3300/0 conn(0x7fbf580b5710 0x7fbf5814a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:06.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.684+0000 7fbf567fc640 1 -- 172.21.15.5:0/2203710115 >> v1:172.21.15.5:6789/0 conn(0x7fbf58141430 legacy=0x7fbf580b7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:06.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.684+0000 7fbf567fc640 1 -- 172.21.15.5:0/2203710115 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf580b4c30 con 0x7fbf5814ae60 2024-03-18T15:34:06.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.684+0000 7fbf55ffb640 1 -- 172.21.15.5:0/2203710115 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2399411911 0 0) 0x7fbf580b9d50 con 0x7fbf58141430 2024-03-18T15:34:06.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.684+0000 7fbf567fc640 1 --2- 172.21.15.5:0/2203710115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf5814ae60 0x7fbf58140ef0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbf5806c3e0 tx=0x7fbf4c01c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1083d454bbc2f084 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:06.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.684+0000 7fbf55ffb640 1 -- 172.21.15.5:0/2203710115 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c0b0070 con 0x7fbf5814ae60 2024-03-18T15:34:06.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.684+0000 7fbf55ffb640 1 -- 172.21.15.5:0/2203710115 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf4c0ab600 con 0x7fbf5814ae60 2024-03-18T15:34:06.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.684+0000 7fbf55ffb640 1 -- 172.21.15.5:0/2203710115 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c0ab9d0 con 0x7fbf5814ae60 2024-03-18T15:34:06.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.688+0000 7fbf5d63c640 1 -- 172.21.15.5:0/2203710115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf5814ae60 msgr2=0x7fbf58140ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:06.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.688+0000 7fbf5d63c640 1 --2- 172.21.15.5:0/2203710115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf5814ae60 0x7fbf58140ef0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbf5806c3e0 tx=0x7fbf4c01c360 comp rx=0 tx=0).stop 2024-03-18T15:34:06.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.688+0000 7fbf5d63c640 1 -- 172.21.15.5:0/2203710115 shutdown_connections 2024-03-18T15:34:06.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.688+0000 7fbf5d63c640 1 --2- 172.21.15.5:0/2203710115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf5814ae60 0x7fbf58140ef0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:06.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.688+0000 7fbf5d63c640 1 --2- 172.21.15.5:0/2203710115 >> v2:172.21.15.80:3300/0 conn(0x7fbf580b5710 0x7fbf5814a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:06.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.688+0000 7fbf5d63c640 1 -- 172.21.15.5:0/2203710115 >> 172.21.15.5:0/2203710115 conn(0x7fbf580673c0 msgr2=0x7fbf580677d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:06.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.688+0000 7fbf5d63c640 1 -- 172.21.15.5:0/2203710115 shutdown_connections 2024-03-18T15:34:06.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.688+0000 7fbf5d63c640 1 -- 172.21.15.5:0/2203710115 wait complete. 2024-03-18T15:34:06.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.688+0000 7fbf5d63c640 1 Processor -- start 2024-03-18T15:34:06.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.688+0000 7fbf5d63c640 1 -- start start 2024-03-18T15:34:06.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.688+0000 7fbf5d63c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbf580b5710 0x7fbf58134340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:06.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.688+0000 7fbf5d63c640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbf58141430 0x7fbf581e3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:06.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.688+0000 7fbf5d63c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf5814ae60 0x7fbf58132f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:06.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.688+0000 7fbf5d63c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbf58141840 con 0x7fbf5814ae60 2024-03-18T15:34:06.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.688+0000 7fbf5d63c640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5814b2c0 con 0x7fbf580b5710 2024-03-18T15:34:06.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.688+0000 7fbf5d63c640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fbf581419c0 con 0x7fbf58141430 2024-03-18T15:34:06.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.688+0000 7fbf577fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf5814ae60 0x7fbf58132f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:06.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.688+0000 7fbf577fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf5814ae60 0x7fbf58132f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50146/0 (socket says 172.21.15.5:50146) 2024-03-18T15:34:06.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.688+0000 7fbf577fe640 1 -- 172.21.15.5:0/2437284202 learned_addr learned my addr 172.21.15.5:0/2437284202 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:06.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.688+0000 7fbf56ffd640 1 --2- 172.21.15.5:0/2437284202 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbf580b5710 0x7fbf58134340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:06.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.688+0000 7fbf567fc640 1 --2- 172.21.15.5:0/2437284202 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbf58141430 0x7fbf581e3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:06.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.688+0000 7fbf577fe640 1 -- 172.21.15.5:0/2437284202 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbf58141430 msgr2=0x7fbf581e3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:06.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.688+0000 7fbf577fe640 1 --2- 172.21.15.5:0/2437284202 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbf58141430 0x7fbf581e3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:06.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.688+0000 7fbf577fe640 1 -- 172.21.15.5:0/2437284202 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbf580b5710 msgr2=0x7fbf58134340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:06.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.688+0000 7fbf577fe640 1 --2- 172.21.15.5:0/2437284202 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbf580b5710 0x7fbf58134340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:06.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.688+0000 7fbf577fe640 1 -- 172.21.15.5:0/2437284202 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf581e0fe0 con 0x7fbf5814ae60 2024-03-18T15:34:06.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.688+0000 7fbf56ffd640 1 --2- 172.21.15.5:0/2437284202 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbf580b5710 0x7fbf58134340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:06.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.688+0000 7fbf577fe640 1 --2- 172.21.15.5:0/2437284202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf5814ae60 0x7fbf58132f80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbf5000c7a0 tx=0x7fbf5000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:06.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.688+0000 7fbf3f7fe640 1 -- 172.21.15.5:0/2437284202 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf50018070 con 0x7fbf5814ae60 2024-03-18T15:34:06.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.688+0000 7fbf3f7fe640 1 -- 172.21.15.5:0/2437284202 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf5000a640 con 0x7fbf5814ae60 2024-03-18T15:34:06.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.688+0000 7fbf5d63c640 1 -- 172.21.15.5:0/2437284202 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf581e0500 con 0x7fbf5814ae60 2024-03-18T15:34:06.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.688+0000 7fbf3f7fe640 1 -- 172.21.15.5:0/2437284202 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf5000aab0 con 0x7fbf5814ae60 2024-03-18T15:34:06.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.688+0000 7fbf5d63c640 1 -- 172.21.15.5:0/2437284202 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf581e00b0 con 0x7fbf5814ae60 2024-03-18T15:34:06.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.688+0000 7fbf5d63c640 1 -- 172.21.15.5:0/2437284202 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf5804f550 con 0x7fbf5814ae60 2024-03-18T15:34:06.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.692+0000 7fbf3f7fe640 1 -- 172.21.15.5:0/2437284202 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fbf5001b070 con 0x7fbf5814ae60 2024-03-18T15:34:06.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.692+0000 7fbf3f7fe640 1 --2- 172.21.15.5:0/2437284202 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fbf44067660 0x7fbf44069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:06.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.692+0000 7fbf56ffd640 1 --2- 172.21.15.5:0/2437284202 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fbf44067660 0x7fbf44069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:06.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.692+0000 7fbf3f7fe640 1 -- 172.21.15.5:0/2437284202 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fbf50084dd0 con 0x7fbf5814ae60 2024-03-18T15:34:06.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.692+0000 7fbf56ffd640 1 --2- 172.21.15.5:0/2437284202 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fbf44067660 0x7fbf44069b20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbf580b6790 tx=0x7fbf48008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:06.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.696+0000 7fbf3f7fe640 1 -- 172.21.15.5:0/2437284202 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fbf50055ae0 con 0x7fbf5814ae60 2024-03-18T15:34:06.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.856+0000 7fbf5d63c640 1 -- 172.21.15.5:0/2437284202 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbf580b7140 con 0x7fbf5814ae60 2024-03-18T15:34:06.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.856+0000 7fbf3f7fe640 1 -- 172.21.15.5:0/2437284202 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7fbf500553e0 con 0x7fbf5814ae60 2024-03-18T15:34:06.861 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-03-18T15:34:06.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.860+0000 7fbf5d63c640 1 -- 172.21.15.5:0/2437284202 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fbf44067660 msgr2=0x7fbf44069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:06.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.860+0000 7fbf5d63c640 1 --2- 172.21.15.5:0/2437284202 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fbf44067660 0x7fbf44069b20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbf580b6790 tx=0x7fbf48008040 comp rx=0 tx=0).stop 2024-03-18T15:34:06.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.860+0000 7fbf5d63c640 1 -- 172.21.15.5:0/2437284202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf5814ae60 msgr2=0x7fbf58132f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:06.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.860+0000 7fbf5d63c640 1 --2- 172.21.15.5:0/2437284202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf5814ae60 0x7fbf58132f80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbf5000c7a0 tx=0x7fbf5000cc60 comp rx=0 tx=0).stop 2024-03-18T15:34:06.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.860+0000 7fbf5d63c640 1 -- 172.21.15.5:0/2437284202 shutdown_connections 2024-03-18T15:34:06.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.860+0000 7fbf5d63c640 1 --2- 172.21.15.5:0/2437284202 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fbf44067660 0x7fbf44069b20 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:06.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.860+0000 7fbf5d63c640 1 --2- 172.21.15.5:0/2437284202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf5814ae60 0x7fbf58132f80 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:06.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.860+0000 7fbf5d63c640 1 --2- 172.21.15.5:0/2437284202 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbf58141430 0x7fbf581e3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:06.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.860+0000 7fbf5d63c640 1 --2- 172.21.15.5:0/2437284202 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbf580b5710 0x7fbf58134340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:06.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.860+0000 7fbf5d63c640 1 -- 172.21.15.5:0/2437284202 >> 172.21.15.5:0/2437284202 conn(0x7fbf580673c0 msgr2=0x7fbf581452a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:06.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.860+0000 7fbf5d63c640 1 -- 172.21.15.5:0/2437284202 shutdown_connections 2024-03-18T15:34:06.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:06.860+0000 7fbf5d63c640 1 -- 172.21.15.5:0/2437284202 wait complete. 2024-03-18T15:34:06.881 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-18T15:34:07.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.004+0000 7fef19e92640 1 Processor -- start 2024-03-18T15:34:07.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.004+0000 7fef19e92640 1 -- start start 2024-03-18T15:34:07.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.004+0000 7fef19e92640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fef1406b4e0 0x7fef140702b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:07.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.004+0000 7fef19e92640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fef14141360 0x7fef141417e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:07.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.004+0000 7fef19e92640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef1406b8c0 con 0x7fef141474c0 2024-03-18T15:34:07.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.004+0000 7fef19e92640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fef1406c0e0 con 0x7fef1406b4e0 2024-03-18T15:34:07.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.004+0000 7fef19e92640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fef140719b0 con 0x7fef14141360 2024-03-18T15:34:07.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.004+0000 7fef137fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fef1406b4e0 0x7fef140702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:07.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.004+0000 7fef137fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fef1406b4e0 0x7fef140702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50160/0 (socket says 172.21.15.5:50160) 2024-03-18T15:34:07.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.004+0000 7fef137fe640 1 -- 172.21.15.5:0/3399778327 learned_addr learned my addr 172.21.15.5:0/3399778327 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:07.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.004+0000 7fef0bfff640 1 --2- 172.21.15.5:0/3399778327 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fef14141360 0x7fef141417e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:07.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.004+0000 7fef137fe640 1 -- 172.21.15.5:0/3399778327 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fef14141360 msgr2=0x7fef141417e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:07.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.004+0000 7fef137fe640 1 --2- 172.21.15.5:0/3399778327 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fef14141360 0x7fef141417e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:07.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.004+0000 7fef137fe640 1 -- 172.21.15.5:0/3399778327 >> v1:172.21.15.80:6789/0 conn(0x7fef141474c0 legacy=0x7fef1414aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:07.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.004+0000 7fef137fe640 1 -- 172.21.15.5:0/3399778327 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef1406e800 con 0x7fef1406b4e0 2024-03-18T15:34:07.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.008+0000 7fef137fe640 1 --2- 172.21.15.5:0/3399778327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef1406b4e0 0x7fef140702b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fef04009870 tx=0x7fef0401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bed83fd3d32ec03b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:07.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.008+0000 7fef12ffd640 1 -- 172.21.15.5:0/3399778327 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef040b0070 con 0x7fef1406b4e0 2024-03-18T15:34:07.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.008+0000 7fef12ffd640 1 -- 172.21.15.5:0/3399778327 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef040ab600 con 0x7fef1406b4e0 2024-03-18T15:34:07.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.008+0000 7fef12ffd640 1 -- 172.21.15.5:0/3399778327 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef040ab9d0 con 0x7fef1406b4e0 2024-03-18T15:34:07.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.008+0000 7fef19e92640 1 -- 172.21.15.5:0/3399778327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef1406b4e0 msgr2=0x7fef140702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:07.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.008+0000 7fef19e92640 1 --2- 172.21.15.5:0/3399778327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef1406b4e0 0x7fef140702b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fef04009870 tx=0x7fef0401c4d0 comp rx=0 tx=0).stop 2024-03-18T15:34:07.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.008+0000 7fef19e92640 1 -- 172.21.15.5:0/3399778327 shutdown_connections 2024-03-18T15:34:07.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.008+0000 7fef19e92640 1 --2- 172.21.15.5:0/3399778327 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fef14141360 0x7fef141417e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:07.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.008+0000 7fef19e92640 1 --2- 172.21.15.5:0/3399778327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef1406b4e0 0x7fef140702b0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:07.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.008+0000 7fef19e92640 1 -- 172.21.15.5:0/3399778327 >> 172.21.15.5:0/3399778327 conn(0x7fef140b7a80 msgr2=0x7fef140b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:07.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.008+0000 7fef19e92640 1 -- 172.21.15.5:0/3399778327 shutdown_connections 2024-03-18T15:34:07.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.008+0000 7fef19e92640 1 -- 172.21.15.5:0/3399778327 wait complete. 2024-03-18T15:34:07.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.008+0000 7fef19e92640 1 Processor -- start 2024-03-18T15:34:07.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.008+0000 7fef19e92640 1 -- start start 2024-03-18T15:34:07.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.008+0000 7fef19e92640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fef14141360 0x7fef1414a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:07.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.008+0000 7fef19e92640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fef141474c0 0x7fef141493f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:07.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.008+0000 7fef19e92640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef14148620 0x7fef141e3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:07.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.008+0000 7fef19e92640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fef14154f00 con 0x7fef14148620 2024-03-18T15:34:07.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.008+0000 7fef19e92640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fef14154d80 con 0x7fef141474c0 2024-03-18T15:34:07.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.008+0000 7fef19e92640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fef14155080 con 0x7fef14141360 2024-03-18T15:34:07.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.008+0000 7fef13fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef14148620 0x7fef141e3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:07.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.012+0000 7fef13fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef14148620 0x7fef141e3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50168/0 (socket says 172.21.15.5:50168) 2024-03-18T15:34:07.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.012+0000 7fef13fff640 1 -- 172.21.15.5:0/2920976813 learned_addr learned my addr 172.21.15.5:0/2920976813 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:07.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.012+0000 7fef137fe640 1 --2- 172.21.15.5:0/2920976813 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fef14141360 0x7fef1414a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:07.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.012+0000 7fef0bfff640 1 --2- 172.21.15.5:0/2920976813 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fef141474c0 0x7fef141493f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:07.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.012+0000 7fef13fff640 1 -- 172.21.15.5:0/2920976813 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fef14141360 msgr2=0x7fef1414a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:07.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.012+0000 7fef13fff640 1 --2- 172.21.15.5:0/2920976813 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fef14141360 0x7fef1414a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:07.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.012+0000 7fef13fff640 1 -- 172.21.15.5:0/2920976813 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fef141474c0 msgr2=0x7fef141493f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:07.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.012+0000 7fef13fff640 1 --2- 172.21.15.5:0/2920976813 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fef141474c0 0x7fef141493f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:07.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.012+0000 7fef13fff640 1 -- 172.21.15.5:0/2920976813 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef14149930 con 0x7fef14148620 2024-03-18T15:34:07.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.012+0000 7fef13fff640 1 --2- 172.21.15.5:0/2920976813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef14148620 0x7fef141e3ac0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fef0c00c7a0 tx=0x7fef0c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:07.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.012+0000 7fef10ff9640 1 -- 172.21.15.5:0/2920976813 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0c003b80 con 0x7fef14148620 2024-03-18T15:34:07.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.012+0000 7fef10ff9640 1 -- 172.21.15.5:0/2920976813 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef0c003d20 con 0x7fef14148620 2024-03-18T15:34:07.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.012+0000 7fef19e92640 1 -- 172.21.15.5:0/2920976813 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef14149bb0 con 0x7fef14148620 2024-03-18T15:34:07.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.012+0000 7fef10ff9640 1 -- 172.21.15.5:0/2920976813 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0c00aa30 con 0x7fef14148620 2024-03-18T15:34:07.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.012+0000 7fef19e92640 1 -- 172.21.15.5:0/2920976813 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef141e35a0 con 0x7fef14148620 2024-03-18T15:34:07.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.012+0000 7fef19e92640 1 -- 172.21.15.5:0/2920976813 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef1404f550 con 0x7fef14148620 2024-03-18T15:34:07.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.012+0000 7fef10ff9640 1 -- 172.21.15.5:0/2920976813 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fef0c00abd0 con 0x7fef14148620 2024-03-18T15:34:07.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.012+0000 7fef10ff9640 1 --2- 172.21.15.5:0/2920976813 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7feef4067660 0x7feef4069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:07.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.012+0000 7fef10ff9640 1 -- 172.21.15.5:0/2920976813 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fef0c085fa0 con 0x7fef14148620 2024-03-18T15:34:07.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.012+0000 7fef137fe640 1 --2- 172.21.15.5:0/2920976813 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7feef4067660 0x7feef4069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:07.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.012+0000 7fef137fe640 1 --2- 172.21.15.5:0/2920976813 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7feef4067660 0x7feef4069b20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fef040ad000 tx=0x7fef040a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:07.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.016+0000 7fef10ff9640 1 -- 172.21.15.5:0/2920976813 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fef14149930 con 0x7fef14148620 2024-03-18T15:34:07.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.192+0000 7fef19e92640 1 -- 172.21.15.5:0/2920976813 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fef1406c9a0 con 0x7fef14148620 2024-03-18T15:34:07.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.192+0000 7fef10ff9640 1 -- 172.21.15.5:0/2920976813 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7fef0c056740 con 0x7fef14148620 2024-03-18T15:34:07.195 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2024-03-18T15:34:07.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.196+0000 7fef19e92640 1 -- 172.21.15.5:0/2920976813 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7feef4067660 msgr2=0x7feef4069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:07.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.196+0000 7fef19e92640 1 --2- 172.21.15.5:0/2920976813 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7feef4067660 0x7feef4069b20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fef040ad000 tx=0x7fef040a6000 comp rx=0 tx=0).stop 2024-03-18T15:34:07.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.196+0000 7fef19e92640 1 -- 172.21.15.5:0/2920976813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef14148620 msgr2=0x7fef141e3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:07.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.196+0000 7fef19e92640 1 --2- 172.21.15.5:0/2920976813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef14148620 0x7fef141e3ac0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fef0c00c7a0 tx=0x7fef0c00cc60 comp rx=0 tx=0).stop 2024-03-18T15:34:07.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.196+0000 7fef19e92640 1 -- 172.21.15.5:0/2920976813 shutdown_connections 2024-03-18T15:34:07.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.196+0000 7fef19e92640 1 --2- 172.21.15.5:0/2920976813 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7feef4067660 0x7feef4069b20 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:07.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.196+0000 7fef19e92640 1 --2- 172.21.15.5:0/2920976813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef14148620 0x7fef141e3ac0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:07.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.196+0000 7fef19e92640 1 --2- 172.21.15.5:0/2920976813 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fef141474c0 0x7fef141493f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:07.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.196+0000 7fef19e92640 1 --2- 172.21.15.5:0/2920976813 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fef14141360 0x7fef1414a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:07.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.196+0000 7fef19e92640 1 -- 172.21.15.5:0/2920976813 >> 172.21.15.5:0/2920976813 conn(0x7fef140b7a80 msgr2=0x7fef14072620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:07.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.196+0000 7fef19e92640 1 -- 172.21.15.5:0/2920976813 shutdown_connections 2024-03-18T15:34:07.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.196+0000 7fef19e92640 1 -- 172.21.15.5:0/2920976813 wait complete. 2024-03-18T15:34:07.214 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-03-18T15:34:07.215 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-18T15:34:07.220 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:34:07.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.384+0000 7fad7bc99640 1 Processor -- start 2024-03-18T15:34:07.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.384+0000 7fad7bc99640 1 -- start start 2024-03-18T15:34:07.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.388+0000 7fad7bc99640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fad740b5710 0x7fad7414a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:07.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.388+0000 7fad7bc99640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fad74141600 0x7fad74151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:07.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.388+0000 7fad7bc99640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad740b9d50 con 0x7fad7414ae60 2024-03-18T15:34:07.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.388+0000 7fad7bc99640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fad74141a90 con 0x7fad74141600 2024-03-18T15:34:07.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.388+0000 7fad7bc99640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fad740b4320 con 0x7fad740b5710 2024-03-18T15:34:07.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.388+0000 7fad7a236640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fad74141600 0x7fad74151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:07.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.388+0000 7fad7a236640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fad74141600 0x7fad74151cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50176/0 (socket says 172.21.15.5:50176) 2024-03-18T15:34:07.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.388+0000 7fad7a236640 1 -- 172.21.15.5:0/945843464 learned_addr learned my addr 172.21.15.5:0/945843464 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:07.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.388+0000 7fad79a35640 1 --2- 172.21.15.5:0/945843464 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fad740b5710 0x7fad7414a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:07.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.388+0000 7fad7a236640 1 -- 172.21.15.5:0/945843464 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fad740b5710 msgr2=0x7fad7414a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:07.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.388+0000 7fad7a236640 1 --2- 172.21.15.5:0/945843464 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fad740b5710 0x7fad7414a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:07.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.388+0000 7fad7a236640 1 -- 172.21.15.5:0/945843464 >> v1:172.21.15.80:6789/0 conn(0x7fad7414ae60 legacy=0x7fad74140ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:07.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.388+0000 7fad7a236640 1 -- 172.21.15.5:0/945843464 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad740b4c30 con 0x7fad74141600 2024-03-18T15:34:07.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.388+0000 7fad7a236640 1 --2- 172.21.15.5:0/945843464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad74141600 0x7fad74151cb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fad6c009e80 tx=0x7fad6c01c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2185a4db93fe9fa2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:07.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.388+0000 7fad78a33640 1 -- 172.21.15.5:0/945843464 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c0a55b0 con 0x7fad74141600 2024-03-18T15:34:07.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.388+0000 7fad78a33640 1 -- 172.21.15.5:0/945843464 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad6c0a5750 con 0x7fad74141600 2024-03-18T15:34:07.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.388+0000 7fad78a33640 1 -- 172.21.15.5:0/945843464 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c0a5b20 con 0x7fad74141600 2024-03-18T15:34:07.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.388+0000 7fad7bc99640 1 -- 172.21.15.5:0/945843464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad74141600 msgr2=0x7fad74151cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:07.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.388+0000 7fad7bc99640 1 --2- 172.21.15.5:0/945843464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad74141600 0x7fad74151cb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fad6c009e80 tx=0x7fad6c01c8f0 comp rx=0 tx=0).stop 2024-03-18T15:34:07.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.392+0000 7fad7bc99640 1 -- 172.21.15.5:0/945843464 shutdown_connections 2024-03-18T15:34:07.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.392+0000 7fad7bc99640 1 --2- 172.21.15.5:0/945843464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad74141600 0x7fad74151cb0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:07.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.392+0000 7fad7bc99640 1 --2- 172.21.15.5:0/945843464 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fad740b5710 0x7fad7414a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:07.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.392+0000 7fad7bc99640 1 -- 172.21.15.5:0/945843464 >> 172.21.15.5:0/945843464 conn(0x7fad740673c0 msgr2=0x7fad740677d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:07.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.392+0000 7fad7bc99640 1 -- 172.21.15.5:0/945843464 shutdown_connections 2024-03-18T15:34:07.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.392+0000 7fad7bc99640 1 -- 172.21.15.5:0/945843464 wait complete. 2024-03-18T15:34:07.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.392+0000 7fad7bc99640 1 Processor -- start 2024-03-18T15:34:07.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.392+0000 7fad7bc99640 1 -- start start 2024-03-18T15:34:07.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.392+0000 7fad7bc99640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fad740b5710 0x7fad740bb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:07.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.392+0000 7fad7bc99640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad74141600 0x7fad741e3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:07.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.392+0000 7fad7bc99640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad7414ae60 0x7fad741e3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:07.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.392+0000 7fad7bc99640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fad740b50e0 con 0x7fad7414ae60 2024-03-18T15:34:07.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.392+0000 7fad7bc99640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fad740b6940 con 0x7fad74141600 2024-03-18T15:34:07.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.392+0000 7fad7bc99640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fad740b5260 con 0x7fad740b5710 2024-03-18T15:34:07.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.392+0000 7fad7a236640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad7414ae60 0x7fad741e3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:07.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.392+0000 7fad7a236640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad7414ae60 0x7fad741e3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50178/0 (socket says 172.21.15.5:50178) 2024-03-18T15:34:07.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.392+0000 7fad7a236640 1 -- 172.21.15.5:0/1253614420 learned_addr learned my addr 172.21.15.5:0/1253614420 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:07.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.392+0000 7fad79a35640 1 --2- 172.21.15.5:0/1253614420 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fad740b5710 0x7fad740bb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:07.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.392+0000 7fad79234640 1 --2- 172.21.15.5:0/1253614420 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad74141600 0x7fad741e3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:07.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.392+0000 7fad7a236640 1 -- 172.21.15.5:0/1253614420 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fad740b5710 msgr2=0x7fad740bb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:07.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.392+0000 7fad7a236640 1 --2- 172.21.15.5:0/1253614420 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fad740b5710 0x7fad740bb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:07.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.392+0000 7fad7a236640 1 -- 172.21.15.5:0/1253614420 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad74141600 msgr2=0x7fad741e3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:07.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.392+0000 7fad7a236640 1 --2- 172.21.15.5:0/1253614420 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad74141600 0x7fad741e3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:07.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.392+0000 7fad7a236640 1 -- 172.21.15.5:0/1253614420 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad740bb140 con 0x7fad7414ae60 2024-03-18T15:34:07.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.392+0000 7fad7a236640 1 --2- 172.21.15.5:0/1253614420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad7414ae60 0x7fad741e3890 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fad6c01afa0 tx=0x7fad6c019690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:07.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.392+0000 7fad6a7fc640 1 -- 172.21.15.5:0/1253614420 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c0bc020 con 0x7fad7414ae60 2024-03-18T15:34:07.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.392+0000 7fad6a7fc640 1 -- 172.21.15.5:0/1253614420 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad6c0b8690 con 0x7fad7414ae60 2024-03-18T15:34:07.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.392+0000 7fad6a7fc640 1 -- 172.21.15.5:0/1253614420 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c009510 con 0x7fad7414ae60 2024-03-18T15:34:07.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.392+0000 7fad7bc99640 1 -- 172.21.15.5:0/1253614420 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad740bd030 con 0x7fad7414ae60 2024-03-18T15:34:07.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.392+0000 7fad7bc99640 1 -- 172.21.15.5:0/1253614420 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad7413a130 con 0x7fad7414ae60 2024-03-18T15:34:07.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.396+0000 7fad7bc99640 1 -- 172.21.15.5:0/1253614420 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad7404f550 con 0x7fad7414ae60 2024-03-18T15:34:07.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.396+0000 7fad6a7fc640 1 -- 172.21.15.5:0/1253614420 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fad6c0bf020 con 0x7fad7414ae60 2024-03-18T15:34:07.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.396+0000 7fad6a7fc640 1 --2- 172.21.15.5:0/1253614420 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fad58067690 0x7fad58069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:07.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.396+0000 7fad6a7fc640 1 -- 172.21.15.5:0/1253614420 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fad6c120410 con 0x7fad7414ae60 2024-03-18T15:34:07.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.396+0000 7fad79a35640 1 --2- 172.21.15.5:0/1253614420 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fad58067690 0x7fad58069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:07.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.396+0000 7fad79a35640 1 --2- 172.21.15.5:0/1253614420 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fad58067690 0x7fad58069b50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fad7000b930 tx=0x7fad70008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:07.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.400+0000 7fad6a7fc640 1 -- 172.21.15.5:0/1253614420 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fad6c0c6070 con 0x7fad7414ae60 2024-03-18T15:34:07.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.564+0000 7fad7bc99640 1 -- 172.21.15.5:0/1253614420 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad740b7140 con 0x7fad7414ae60 2024-03-18T15:34:07.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.564+0000 7fad6a7fc640 1 -- 172.21.15.5:0/1253614420 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16449 (secure 0 0 0) 0x7fad6c016600 con 0x7fad7414ae60 2024-03-18T15:34:07.568 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:34:07.568 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":21,"fsid":"ef54fe03-5aa7-4eb2-9b1e-66c9cc345b94","created":"2024-03-18T15:33:18.086614+0000","modified":"2024-03-18T15:33:43.984050+0000","last_up_change":"2024-03-18T15:33:29.229911+0000","last_in_change":"2024-03-18T15:33:20.323840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:28.963186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T15:33:40.533646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T15:33:41.323519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df1c754b-af59-4d16-8919-fbb6386b3c48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6804","nonce":976264754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6808","nonce":976264754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6815","nonce":976264754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6811","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6813","nonce":976264754}]},"public_addr":"172.21.15.5:6804/976264754","cluster_addr":"172.21.15.5:6808/976264754","heartbeat_back_addr":"172.21.15.5:6815/976264754","heartbeat_front_addr":"172.21.15.5:6813/976264754","state":["exists","up"]},{"osd":1,"uuid":"b9a08b5e-8af4-4144-bdba-d69ed1ac627a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6801","nonce":3901761126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6803","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6805","nonce":3901761126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6812","nonce":3901761126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6807","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6809","nonce":3901761126}]},"public_addr":"172.21.15.5:6801/3901761126","cluster_addr":"172.21.15.5:6805/3901761126","heartbeat_back_addr":"172.21.15.5:6812/3901761126","heartbeat_front_addr":"172.21.15.5:6809/3901761126","state":["exists","up"]},{"osd":2,"uuid":"c3e8bb76-e382-439d-b380-62ef92f1255a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6817","nonce":502968448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6819","nonce":502968448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6823","nonce":502968448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6821","nonce":502968448}]},"public_addr":"172.21.15.5:6817/502968448","cluster_addr":"172.21.15.5:6819/502968448","heartbeat_back_addr":"172.21.15.5:6823/502968448","heartbeat_front_addr":"172.21.15.5:6821/502968448","state":["exists","up"]},{"osd":3,"uuid":"e8c94625-f2cd-4eba-b53d-2e7356e1c02a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6825","nonce":2425932735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6827","nonce":2425932735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6831","nonce":2425932735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6829","nonce":2425932735}]},"public_addr":"172.21.15.5:6825/2425932735","cluster_addr":"172.21.15.5:6827/2425932735","heartbeat_back_addr":"172.21.15.5:6831/2425932735","heartbeat_front_addr":"172.21.15.5:6829/2425932735","state":["exists","up"]},{"osd":4,"uuid":"0099dd52-6265-4d32-82e4-3ef16e73add0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6817","nonce":976514285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6819","nonce":976514285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6823","nonce":976514285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6821","nonce":976514285}]},"public_addr":"172.21.15.80:6817/976514285","cluster_addr":"172.21.15.80:6819/976514285","heartbeat_back_addr":"172.21.15.80:6823/976514285","heartbeat_front_addr":"172.21.15.80:6821/976514285","state":["exists","up"]},{"osd":5,"uuid":"ed10c84f-8c39-41cc-9b64-7c482a94a48e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6825","nonce":627153447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6827","nonce":627153447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6831","nonce":627153447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6829","nonce":627153447}]},"public_addr":"172.21.15.80:6825/627153447","cluster_addr":"172.21.15.80:6827/627153447","heartbeat_back_addr":"172.21.15.80:6831/627153447","heartbeat_front_addr":"172.21.15.80:6829/627153447","state":["exists","up"]},{"osd":6,"uuid":"c7a3538a-7048-4fba-8781-b440f03c25db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6803","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6805","nonce":488342527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6807","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6809","nonce":488342527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6815","nonce":488342527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6811","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6813","nonce":488342527}]},"public_addr":"172.21.15.80:6805/488342527","cluster_addr":"172.21.15.80:6809/488342527","heartbeat_back_addr":"172.21.15.80:6815/488342527","heartbeat_front_addr":"172.21.15.80:6813/488342527","state":["exists","up"]},{"osd":7,"uuid":"f9418104-3a97-4110-bc3a-1f8151d0f757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6801","nonce":982831666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6804","nonce":982831666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6812","nonce":982831666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6808","nonce":982831666}]},"public_addr":"172.21.15.80:6801/982831666","cluster_addr":"172.21.15.80:6804/982831666","heartbeat_back_addr":"172.21.15.80:6812/982831666","heartbeat_front_addr":"172.21.15.80:6808/982831666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.970828+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.858116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.987556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.864603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.756293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.591502+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:34:07.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.568+0000 7fad7bc99640 1 -- 172.21.15.5:0/1253614420 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fad58067690 msgr2=0x7fad58069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:07.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.568+0000 7fad7bc99640 1 --2- 172.21.15.5:0/1253614420 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fad58067690 0x7fad58069b50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fad7000b930 tx=0x7fad70008040 comp rx=0 tx=0).stop 2024-03-18T15:34:07.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.568+0000 7fad7bc99640 1 -- 172.21.15.5:0/1253614420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad7414ae60 msgr2=0x7fad741e3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:07.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.568+0000 7fad7bc99640 1 --2- 172.21.15.5:0/1253614420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad7414ae60 0x7fad741e3890 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fad6c01afa0 tx=0x7fad6c019690 comp rx=0 tx=0).stop 2024-03-18T15:34:07.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.568+0000 7fad7bc99640 1 -- 172.21.15.5:0/1253614420 shutdown_connections 2024-03-18T15:34:07.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.568+0000 7fad7bc99640 1 --2- 172.21.15.5:0/1253614420 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fad58067690 0x7fad58069b50 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:07.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.568+0000 7fad7bc99640 1 --2- 172.21.15.5:0/1253614420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad7414ae60 0x7fad741e3890 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:07.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.568+0000 7fad7bc99640 1 --2- 172.21.15.5:0/1253614420 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad74141600 0x7fad741e3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:07.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.568+0000 7fad7bc99640 1 --2- 172.21.15.5:0/1253614420 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fad740b5710 0x7fad740bb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:07.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.568+0000 7fad7bc99640 1 -- 172.21.15.5:0/1253614420 >> 172.21.15.5:0/1253614420 conn(0x7fad740673c0 msgr2=0x7fad741473a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:07.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.568+0000 7fad7bc99640 1 -- 172.21.15.5:0/1253614420 shutdown_connections 2024-03-18T15:34:07.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.568+0000 7fad7bc99640 1 -- 172.21.15.5:0/1253614420 wait complete. 2024-03-18T15:34:07.588 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T15:33:28.963186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-18T15:33:40.533646+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-18T15:33:41.323519+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T15:34:07.588 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T15:34:07.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.716+0000 7fa6bf6d2640 1 Processor -- start 2024-03-18T15:34:07.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.716+0000 7fa6bf6d2640 1 -- start start 2024-03-18T15:34:07.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.716+0000 7fa6bf6d2640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa6b80ade00 0x7fa6b8147700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:07.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.716+0000 7fa6bf6d2640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6b814ba90 con 0x7fa6b80ae5b0 2024-03-18T15:34:07.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.716+0000 7fa6bf6d2640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6b8074b80 con 0x7fa6b80747a0 2024-03-18T15:34:07.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.716+0000 7fa6bf6d2640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fa6b8147c40 con 0x7fa6b80ade00 2024-03-18T15:34:07.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.716+0000 7fa6bcc6d640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa6b80ae5b0 0x7fa6b814b380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60124/0 (socket says 172.21.15.5:60124) 2024-03-18T15:34:07.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.716+0000 7fa6bcc6d640 1 -- 172.21.15.5:0/854607538 learned_addr learned my addr 172.21.15.5:0/854607538 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:07.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.716+0000 7fa6bdc6f640 1 --2- 172.21.15.5:0/854607538 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa6b80ade00 0x7fa6b8147700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:07.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.716+0000 7fa6affff640 1 -- 172.21.15.5:0/854607538 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2401524629 0 0) 0x7fa6b814ba90 con 0x7fa6b80ae5b0 2024-03-18T15:34:07.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.716+0000 7fa6affff640 1 -- 172.21.15.5:0/854607538 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa6a4003530 con 0x7fa6b80ae5b0 2024-03-18T15:34:07.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.716+0000 7fa6affff640 1 -- 172.21.15.5:0/854607538 <== mon.1 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2092845403 0 0) 0x7fa6b8074b80 con 0x7fa6b80747a0 2024-03-18T15:34:07.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.716+0000 7fa6affff640 1 -- 172.21.15.5:0/854607538 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa6b814ba90 con 0x7fa6b80747a0 2024-03-18T15:34:07.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.716+0000 7fa6affff640 1 -- 172.21.15.5:0/854607538 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3331193349 0 0) 0x7fa6a4003530 con 0x7fa6b80ae5b0 2024-03-18T15:34:07.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.716+0000 7fa6bdc6f640 1 -- 172.21.15.5:0/854607538 >> v1:172.21.15.80:6789/0 conn(0x7fa6b80747a0 legacy=0x7fa6b814ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:07.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.716+0000 7fa6bdc6f640 1 -- 172.21.15.5:0/854607538 >> v1:172.21.15.5:6789/0 conn(0x7fa6b80ae5b0 legacy=0x7fa6b814b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:07.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.716+0000 7fa6bdc6f640 1 -- 172.21.15.5:0/854607538 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6b80aed00 con 0x7fa6b80ade00 2024-03-18T15:34:07.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.720+0000 7fa6bdc6f640 1 --2- 172.21.15.5:0/854607538 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa6b80ade00 0x7fa6b8147700 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0009dc0 tx=0x7fa6b001c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b787e3e196ed766 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:07.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.720+0000 7fa6affff640 1 -- 172.21.15.5:0/854607538 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b00b2070 con 0x7fa6b80ade00 2024-03-18T15:34:07.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.720+0000 7fa6affff640 1 -- 172.21.15.5:0/854607538 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6b00a5550 con 0x7fa6b80ade00 2024-03-18T15:34:07.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.720+0000 7fa6affff640 1 -- 172.21.15.5:0/854607538 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a8001680 con 0x7fa6b80ade00 2024-03-18T15:34:07.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.720+0000 7fa6bf6d2640 1 -- 172.21.15.5:0/854607538 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa6b80ade00 msgr2=0x7fa6b8147700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:07.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.720+0000 7fa6bf6d2640 1 --2- 172.21.15.5:0/854607538 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa6b80ade00 0x7fa6b8147700 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0009dc0 tx=0x7fa6b001c820 comp rx=0 tx=0).stop 2024-03-18T15:34:07.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.720+0000 7fa6bf6d2640 1 -- 172.21.15.5:0/854607538 shutdown_connections 2024-03-18T15:34:07.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.720+0000 7fa6bf6d2640 1 --2- 172.21.15.5:0/854607538 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa6b80ade00 0x7fa6b8147700 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:07.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.720+0000 7fa6bf6d2640 1 -- 172.21.15.5:0/854607538 >> 172.21.15.5:0/854607538 conn(0x7fa6b80b5d70 msgr2=0x7fa6b80b6180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:07.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.720+0000 7fa6bf6d2640 1 -- 172.21.15.5:0/854607538 shutdown_connections 2024-03-18T15:34:07.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.720+0000 7fa6bf6d2640 1 -- 172.21.15.5:0/854607538 wait complete. 2024-03-18T15:34:07.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.720+0000 7fa6bf6d2640 1 Processor -- start 2024-03-18T15:34:07.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.720+0000 7fa6bf6d2640 1 -- start start 2024-03-18T15:34:07.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.724+0000 7fa6bf6d2640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa6b80747a0 0x7fa6b8140610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:07.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.724+0000 7fa6bf6d2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6b80ade00 0x7fa6b80b2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:07.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.724+0000 7fa6bf6d2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6b80ae5b0 0x7fa6b81e0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:07.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.724+0000 7fa6bf6d2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b80b8d00 con 0x7fa6b80ade00 2024-03-18T15:34:07.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.724+0000 7fa6bf6d2640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b80b8b80 con 0x7fa6b80ae5b0 2024-03-18T15:34:07.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.724+0000 7fa6bf6d2640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fa6b8155a50 con 0x7fa6b80747a0 2024-03-18T15:34:07.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.724+0000 7fa6bcc6d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6b80ade00 0x7fa6b80b2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:07.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.724+0000 7fa6bcc6d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6b80ade00 0x7fa6b80b2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50192/0 (socket says 172.21.15.5:50192) 2024-03-18T15:34:07.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.724+0000 7fa6bcc6d640 1 -- 172.21.15.5:0/693137922 learned_addr learned my addr 172.21.15.5:0/693137922 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:07.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.724+0000 7fa6bd46e640 1 --2- 172.21.15.5:0/693137922 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa6b80747a0 0x7fa6b8140610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:07.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.724+0000 7fa6bdc6f640 1 --2- 172.21.15.5:0/693137922 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6b80ae5b0 0x7fa6b81e0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:07.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.724+0000 7fa6bcc6d640 1 -- 172.21.15.5:0/693137922 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa6b80747a0 msgr2=0x7fa6b8140610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:07.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.724+0000 7fa6bcc6d640 1 --2- 172.21.15.5:0/693137922 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa6b80747a0 0x7fa6b8140610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:07.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.724+0000 7fa6bcc6d640 1 -- 172.21.15.5:0/693137922 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6b80ae5b0 msgr2=0x7fa6b81e0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:07.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.724+0000 7fa6bcc6d640 1 --2- 172.21.15.5:0/693137922 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6b80ae5b0 0x7fa6b81e0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:07.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.724+0000 7fa6bcc6d640 1 -- 172.21.15.5:0/693137922 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6b81e0500 con 0x7fa6b80ade00 2024-03-18T15:34:07.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.724+0000 7fa6bdc6f640 1 --2- 172.21.15.5:0/693137922 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6b80ae5b0 0x7fa6b81e0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:07.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.724+0000 7fa6bcc6d640 1 --2- 172.21.15.5:0/693137922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6b80ade00 0x7fa6b80b2bc0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8000fe0 tx=0x7fa6a800cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:07.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.724+0000 7fa6adffb640 1 -- 172.21.15.5:0/693137922 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a8017070 con 0x7fa6b80ade00 2024-03-18T15:34:07.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.724+0000 7fa6adffb640 1 -- 172.21.15.5:0/693137922 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6a8005ae0 con 0x7fa6b80ade00 2024-03-18T15:34:07.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.724+0000 7fa6bf6d2640 1 -- 172.21.15.5:0/693137922 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6b80b3300 con 0x7fa6b80ade00 2024-03-18T15:34:07.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.724+0000 7fa6adffb640 1 -- 172.21.15.5:0/693137922 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a8007070 con 0x7fa6b80ade00 2024-03-18T15:34:07.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.724+0000 7fa6bf6d2640 1 -- 172.21.15.5:0/693137922 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6b80b0bc0 con 0x7fa6b80ade00 2024-03-18T15:34:07.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.724+0000 7fa6bf6d2640 1 -- 172.21.15.5:0/693137922 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6b804f550 con 0x7fa6b80ade00 2024-03-18T15:34:07.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.724+0000 7fa6adffb640 1 -- 172.21.15.5:0/693137922 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fa6a801a070 con 0x7fa6b80ade00 2024-03-18T15:34:07.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.724+0000 7fa6adffb640 1 --2- 172.21.15.5:0/693137922 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa69c067660 0x7fa69c069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:07.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.724+0000 7fa6adffb640 1 -- 172.21.15.5:0/693137922 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa6a8084c90 con 0x7fa6b80ade00 2024-03-18T15:34:07.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.724+0000 7fa6bd46e640 1 --2- 172.21.15.5:0/693137922 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa69c067660 0x7fa69c069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:07.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.724+0000 7fa6bd46e640 1 --2- 172.21.15.5:0/693137922 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa69c067660 0x7fa69c069b20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa6b40011f0 tx=0x7fa6b4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:07.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.728+0000 7fa6adffb640 1 -- 172.21.15.5:0/693137922 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fa6a8055350 con 0x7fa6b80ade00 2024-03-18T15:34:07.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.888+0000 7fa6bf6d2640 1 -- 172.21.15.5:0/693137922 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa6b80b5810 con 0x7fa6b80ade00 2024-03-18T15:34:07.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.888+0000 7fa6adffb640 1 -- 172.21.15.5:0/693137922 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7fa6a80593f0 con 0x7fa6b80ade00 2024-03-18T15:34:07.893 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-03-18T15:34:07.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.892+0000 7fa6bf6d2640 1 -- 172.21.15.5:0/693137922 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa69c067660 msgr2=0x7fa69c069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:07.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.892+0000 7fa6bf6d2640 1 --2- 172.21.15.5:0/693137922 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa69c067660 0x7fa69c069b20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa6b40011f0 tx=0x7fa6b4009040 comp rx=0 tx=0).stop 2024-03-18T15:34:07.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.892+0000 7fa6bf6d2640 1 -- 172.21.15.5:0/693137922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6b80ade00 msgr2=0x7fa6b80b2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:07.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.892+0000 7fa6bf6d2640 1 --2- 172.21.15.5:0/693137922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6b80ade00 0x7fa6b80b2bc0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8000fe0 tx=0x7fa6a800cbe0 comp rx=0 tx=0).stop 2024-03-18T15:34:07.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.892+0000 7fa6bf6d2640 1 -- 172.21.15.5:0/693137922 shutdown_connections 2024-03-18T15:34:07.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.892+0000 7fa6bf6d2640 1 --2- 172.21.15.5:0/693137922 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa69c067660 0x7fa69c069b20 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:07.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.892+0000 7fa6bf6d2640 1 --2- 172.21.15.5:0/693137922 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6b80ae5b0 0x7fa6b81e0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:07.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.892+0000 7fa6bf6d2640 1 --2- 172.21.15.5:0/693137922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6b80ade00 0x7fa6b80b2bc0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:07.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.892+0000 7fa6bf6d2640 1 --2- 172.21.15.5:0/693137922 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa6b80747a0 0x7fa6b8140610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:07.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.892+0000 7fa6bf6d2640 1 -- 172.21.15.5:0/693137922 >> 172.21.15.5:0/693137922 conn(0x7fa6b80b5d70 msgr2=0x7fa6b80b77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:07.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.892+0000 7fa6bf6d2640 1 -- 172.21.15.5:0/693137922 shutdown_connections 2024-03-18T15:34:07.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:07.892+0000 7fa6bf6d2640 1 -- 172.21.15.5:0/693137922 wait complete. 2024-03-18T15:34:07.911 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-18T15:34:08.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.040+0000 7f54a9a6c640 1 Processor -- start 2024-03-18T15:34:08.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.040+0000 7f54a9a6c640 1 -- start start 2024-03-18T15:34:08.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.040+0000 7f54a9a6c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f54a40b2db0 0x7f54a406d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:08.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.040+0000 7f54a9a6c640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54a40afb70 con 0x7f54a4072b40 2024-03-18T15:34:08.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.040+0000 7f54a9a6c640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54a40b4a20 con 0x7f54a40af790 2024-03-18T15:34:08.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.040+0000 7f54a9a6c640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f54a406d5b0 con 0x7f54a40b2db0 2024-03-18T15:34:08.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.040+0000 7f54a2ffd640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f54a4072b40 0x7f54a40b26a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60140/0 (socket says 172.21.15.5:60140) 2024-03-18T15:34:08.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.040+0000 7f54a2ffd640 1 -- 172.21.15.5:0/896433438 learned_addr learned my addr 172.21.15.5:0/896433438 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:08.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.040+0000 7f54a3fff640 1 --2- 172.21.15.5:0/896433438 >> v2:172.21.15.80:3300/0 conn(0x7f54a40b2db0 0x7f54a406d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:08.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.040+0000 7f54a27fc640 1 -- 172.21.15.5:0/896433438 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2065072020 0 0) 0x7f54a40afb70 con 0x7f54a4072b40 2024-03-18T15:34:08.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.040+0000 7f54a27fc640 1 -- 172.21.15.5:0/896433438 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f548c003530 con 0x7f54a4072b40 2024-03-18T15:34:08.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.040+0000 7f54a27fc640 1 -- 172.21.15.5:0/896433438 <== mon.1 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3708522464 0 0) 0x7f54a40b4a20 con 0x7f54a40af790 2024-03-18T15:34:08.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.040+0000 7f54a27fc640 1 -- 172.21.15.5:0/896433438 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f54a40afb70 con 0x7f54a40af790 2024-03-18T15:34:08.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.040+0000 7f54a27fc640 1 -- 172.21.15.5:0/896433438 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 145551917 0 0) 0x7f548c003530 con 0x7f54a4072b40 2024-03-18T15:34:08.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.040+0000 7f54a27fc640 1 -- 172.21.15.5:0/896433438 >> v2:172.21.15.80:3300/0 conn(0x7f54a40b2db0 msgr2=0x7f54a406d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:08.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.040+0000 7f54a27fc640 1 --2- 172.21.15.5:0/896433438 >> v2:172.21.15.80:3300/0 conn(0x7f54a40b2db0 0x7f54a406d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:08.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.040+0000 7f54a27fc640 1 -- 172.21.15.5:0/896433438 >> v1:172.21.15.80:6789/0 conn(0x7f54a40af790 legacy=0x7f54a4147780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:08.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.040+0000 7f54a27fc640 1 -- 172.21.15.5:0/896433438 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54a40b4d00 con 0x7f54a4072b40 2024-03-18T15:34:08.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.040+0000 7f54a27fc640 1 -- 172.21.15.5:0/896433438 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3837549003 0 0) 0x7f5498001680 con 0x7f54a4072b40 2024-03-18T15:34:08.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.040+0000 7f54a27fc640 1 -- 172.21.15.5:0/896433438 >> v1:172.21.15.5:6789/0 conn(0x7f54a4072b40 legacy=0x7f54a40b26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:08.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.040+0000 7f54a27fc640 1 --2- 172.21.15.5:0/896433438 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f548c003c80 0x7f548c004080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:08.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.040+0000 7f54a27fc640 1 --2- 172.21.15.5:0/896433438 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f548c004810 0x7f548c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:08.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.040+0000 7f54a27fc640 1 --2- 172.21.15.5:0/896433438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f548c0052e0 0x7f548c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:08.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.040+0000 7f54a27fc640 1 -- 172.21.15.5:0/896433438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f54a406d5b0 con 0x7f548c0052e0 2024-03-18T15:34:08.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.040+0000 7f54a27fc640 1 -- 172.21.15.5:0/896433438 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f548c00c240 con 0x7f548c004810 2024-03-18T15:34:08.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.040+0000 7f54a27fc640 1 -- 172.21.15.5:0/896433438 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f548c00c440 con 0x7f548c003c80 2024-03-18T15:34:08.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.040+0000 7f54a3fff640 1 --2- 172.21.15.5:0/896433438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f548c0052e0 0x7f548c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:08.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.040+0000 7f54a2ffd640 1 --2- 172.21.15.5:0/896433438 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f548c004810 0x7f548c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:08.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.040+0000 7f54a37fe640 1 --2- 172.21.15.5:0/896433438 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f548c003c80 0x7f548c004080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:08.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.040+0000 7f54a3fff640 1 -- 172.21.15.5:0/896433438 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f548c003c80 msgr2=0x7f548c004080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:08.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.040+0000 7f54a3fff640 1 --2- 172.21.15.5:0/896433438 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f548c003c80 0x7f548c004080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:08.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.040+0000 7f54a3fff640 1 -- 172.21.15.5:0/896433438 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f548c004810 msgr2=0x7f548c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:08.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.040+0000 7f54a3fff640 1 --2- 172.21.15.5:0/896433438 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f548c004810 0x7f548c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:08.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.040+0000 7f54a3fff640 1 -- 172.21.15.5:0/896433438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f548c003530 con 0x7f548c0052e0 2024-03-18T15:34:08.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.044+0000 7f54a3fff640 1 -- 172.21.15.5:0/896433438 reap_dead start 2024-03-18T15:34:08.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.044+0000 7f54a3fff640 1 --2- 172.21.15.5:0/896433438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f548c0052e0 0x7f548c00b9d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f549c009fa0 tx=0x7f549c01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e01db5affca4feb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:08.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.044+0000 7f54a27fc640 1 -- 172.21.15.5:0/896433438 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549c0b2070 con 0x7f548c0052e0 2024-03-18T15:34:08.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.044+0000 7f54a27fc640 1 -- 172.21.15.5:0/896433438 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f549c0a5d50 con 0x7f548c0052e0 2024-03-18T15:34:08.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.044+0000 7f54a27fc640 1 -- 172.21.15.5:0/896433438 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549c009bf0 con 0x7f548c0052e0 2024-03-18T15:34:08.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.044+0000 7f54a9a6c640 1 -- 172.21.15.5:0/896433438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f548c0052e0 msgr2=0x7f548c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:08.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.044+0000 7f54a9a6c640 1 --2- 172.21.15.5:0/896433438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f548c0052e0 0x7f548c00b9d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f549c009fa0 tx=0x7f549c01cc80 comp rx=0 tx=0).stop 2024-03-18T15:34:08.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.044+0000 7f54a9a6c640 1 -- 172.21.15.5:0/896433438 shutdown_connections 2024-03-18T15:34:08.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.044+0000 7f54a9a6c640 1 --2- 172.21.15.5:0/896433438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f548c0052e0 0x7f548c00b9d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:08.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.044+0000 7f54a9a6c640 1 -- 172.21.15.5:0/896433438 >> 172.21.15.5:0/896433438 conn(0x7f54a40b5670 msgr2=0x7f54a40b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:08.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.044+0000 7f54a9a6c640 1 -- 172.21.15.5:0/896433438 shutdown_connections 2024-03-18T15:34:08.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.044+0000 7f54a9a6c640 1 -- 172.21.15.5:0/896433438 wait complete. 2024-03-18T15:34:08.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.044+0000 7f54a9a6c640 1 Processor -- start 2024-03-18T15:34:08.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.044+0000 7f54a9a6c640 1 -- start start 2024-03-18T15:34:08.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.044+0000 7f54a9a6c640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f548c0052e0 0x7f54a4071430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:08.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.044+0000 7f54a9a6c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54a41dccf0 0x7f54a41dd150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:08.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.044+0000 7f54a9a6c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f54a41e0260 0x7f54a41e06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:08.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.044+0000 7f54a9a6c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f54a406d1f0 con 0x7f54a41dccf0 2024-03-18T15:34:08.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.044+0000 7f54a9a6c640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f54a406d070 con 0x7f54a41e0260 2024-03-18T15:34:08.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.044+0000 7f54a9a6c640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f54a406d370 con 0x7f548c0052e0 2024-03-18T15:34:08.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.044+0000 7f54a2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54a41dccf0 0x7f54a41dd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:08.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.044+0000 7f54a2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54a41dccf0 0x7f54a41dd150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50198/0 (socket says 172.21.15.5:50198) 2024-03-18T15:34:08.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.044+0000 7f54a2ffd640 1 -- 172.21.15.5:0/408748010 learned_addr learned my addr 172.21.15.5:0/408748010 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:08.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.044+0000 7f54a37fe640 1 --2- 172.21.15.5:0/408748010 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f548c0052e0 0x7f54a4071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:08.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.044+0000 7f54a3fff640 1 --2- 172.21.15.5:0/408748010 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f54a41e0260 0x7f54a41e06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:08.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.044+0000 7f54a2ffd640 1 -- 172.21.15.5:0/408748010 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f548c0052e0 msgr2=0x7f54a4071430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:08.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.044+0000 7f54a2ffd640 1 --2- 172.21.15.5:0/408748010 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f548c0052e0 0x7f54a4071430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:08.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.044+0000 7f54a2ffd640 1 -- 172.21.15.5:0/408748010 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f54a41e0260 msgr2=0x7f54a41e06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:08.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.044+0000 7f54a2ffd640 1 --2- 172.21.15.5:0/408748010 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f54a41e0260 0x7f54a41e06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:08.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.044+0000 7f54a2ffd640 1 -- 172.21.15.5:0/408748010 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54a4072060 con 0x7f54a41dccf0 2024-03-18T15:34:08.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.044+0000 7f54a2ffd640 1 --2- 172.21.15.5:0/408748010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54a41dccf0 0x7f54a41dd150 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f549800e550 tx=0x7f549800ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:08.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.044+0000 7f548bfff640 1 -- 172.21.15.5:0/408748010 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5498012590 con 0x7f54a41dccf0 2024-03-18T15:34:08.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.044+0000 7f548bfff640 1 -- 172.21.15.5:0/408748010 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5498018070 con 0x7f54a41dccf0 2024-03-18T15:34:08.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.044+0000 7f54a9a6c640 1 -- 172.21.15.5:0/408748010 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54a4143eb0 con 0x7f54a41dccf0 2024-03-18T15:34:08.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.044+0000 7f548bfff640 1 -- 172.21.15.5:0/408748010 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5498012970 con 0x7f54a41dccf0 2024-03-18T15:34:08.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.048+0000 7f54a9a6c640 1 -- 172.21.15.5:0/408748010 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54a41448e0 con 0x7f54a41dccf0 2024-03-18T15:34:08.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.048+0000 7f54a9a6c640 1 -- 172.21.15.5:0/408748010 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54a404f550 con 0x7f54a41dccf0 2024-03-18T15:34:08.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.048+0000 7f548bfff640 1 -- 172.21.15.5:0/408748010 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f5498012b10 con 0x7f54a41dccf0 2024-03-18T15:34:08.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.048+0000 7f548bfff640 1 --2- 172.21.15.5:0/408748010 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5490067690 0x7f5490069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:08.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.048+0000 7f548bfff640 1 -- 172.21.15.5:0/408748010 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f54980853d0 con 0x7f54a41dccf0 2024-03-18T15:34:08.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.048+0000 7f54a37fe640 1 --2- 172.21.15.5:0/408748010 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5490067690 0x7f5490069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:08.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.048+0000 7f54a37fe640 1 --2- 172.21.15.5:0/408748010 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5490067690 0x7f5490069b50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5494005630 tx=0x7f5494009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:08.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.052+0000 7f548bfff640 1 -- 172.21.15.5:0/408748010 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f54980560e0 con 0x7f54a41dccf0 2024-03-18T15:34:08.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.216+0000 7f54a9a6c640 1 -- 172.21.15.5:0/408748010 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f54a40b6fd0 con 0x7f54a41dccf0 2024-03-18T15:34:08.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.216+0000 7f548bfff640 1 -- 172.21.15.5:0/408748010 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f54980559e0 con 0x7f54a41dccf0 2024-03-18T15:34:08.221 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-03-18T15:34:08.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.220+0000 7f54a9a6c640 1 -- 172.21.15.5:0/408748010 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5490067690 msgr2=0x7f5490069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:08.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.220+0000 7f54a9a6c640 1 --2- 172.21.15.5:0/408748010 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5490067690 0x7f5490069b50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5494005630 tx=0x7f5494009040 comp rx=0 tx=0).stop 2024-03-18T15:34:08.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.220+0000 7f54a9a6c640 1 -- 172.21.15.5:0/408748010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54a41dccf0 msgr2=0x7f54a41dd150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:08.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.220+0000 7f54a9a6c640 1 --2- 172.21.15.5:0/408748010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54a41dccf0 0x7f54a41dd150 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f549800e550 tx=0x7f549800ea10 comp rx=0 tx=0).stop 2024-03-18T15:34:08.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.220+0000 7f54a9a6c640 1 -- 172.21.15.5:0/408748010 shutdown_connections 2024-03-18T15:34:08.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.220+0000 7f54a9a6c640 1 --2- 172.21.15.5:0/408748010 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5490067690 0x7f5490069b50 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:08.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.220+0000 7f54a9a6c640 1 --2- 172.21.15.5:0/408748010 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f54a41e0260 0x7f54a41e06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:08.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.220+0000 7f54a9a6c640 1 --2- 172.21.15.5:0/408748010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54a41dccf0 0x7f54a41dd150 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:08.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.220+0000 7f54a9a6c640 1 --2- 172.21.15.5:0/408748010 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f548c0052e0 0x7f54a4071430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:08.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.220+0000 7f54a9a6c640 1 -- 172.21.15.5:0/408748010 >> 172.21.15.5:0/408748010 conn(0x7f54a40b5670 msgr2=0x7f54a40aedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:08.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.220+0000 7f54a9a6c640 1 -- 172.21.15.5:0/408748010 shutdown_connections 2024-03-18T15:34:08.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.220+0000 7f54a9a6c640 1 -- 172.21.15.5:0/408748010 wait complete. 2024-03-18T15:34:08.240 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-18T15:34:08.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.364+0000 7f77c7e93640 1 Processor -- start 2024-03-18T15:34:08.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.364+0000 7f77c7e93640 1 -- start start 2024-03-18T15:34:08.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.364+0000 7f77c7e93640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f77c00af790 0x7f77c0072b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:08.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.364+0000 7f77c7e93640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f77c006d070 0x7f77c0147780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:08.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.364+0000 7f77c7e93640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77c00afb70 con 0x7f77c00b26a0 2024-03-18T15:34:08.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.364+0000 7f77c7e93640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f77c00b4a20 con 0x7f77c00af790 2024-03-18T15:34:08.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.364+0000 7f77c7e93640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f77c00b4d00 con 0x7f77c006d070 2024-03-18T15:34:08.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.364+0000 7f77c6430640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f77c00b26a0 0x7f77c00b2b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60148/0 (socket says 172.21.15.5:60148) 2024-03-18T15:34:08.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.364+0000 7f77c6430640 1 -- 172.21.15.5:0/3838620175 learned_addr learned my addr 172.21.15.5:0/3838620175 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:08.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.368+0000 7f77c542e640 1 --2- 172.21.15.5:0/3838620175 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f77c006d070 0x7f77c0147780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:08.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.368+0000 7f77c5c2f640 1 --2- 172.21.15.5:0/3838620175 >> v2:172.21.15.80:3300/0 conn(0x7f77c00af790 0x7f77c0072b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:08.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.368+0000 7f77c4c2d640 1 -- 172.21.15.5:0/3838620175 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2107873975 0 0) 0x7f77c00afb70 con 0x7f77c00b26a0 2024-03-18T15:34:08.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.368+0000 7f77c4c2d640 1 -- 172.21.15.5:0/3838620175 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f77ac003530 con 0x7f77c00b26a0 2024-03-18T15:34:08.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.368+0000 7f77c4c2d640 1 -- 172.21.15.5:0/3838620175 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2850344355 0 0) 0x7f77ac003530 con 0x7f77c00b26a0 2024-03-18T15:34:08.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.368+0000 7f77c4c2d640 1 -- 172.21.15.5:0/3838620175 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f77c006d070 msgr2=0x7f77c0147780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:08.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.368+0000 7f77c4c2d640 1 --2- 172.21.15.5:0/3838620175 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f77c006d070 0x7f77c0147780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:08.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.368+0000 7f77c4c2d640 1 -- 172.21.15.5:0/3838620175 >> v2:172.21.15.80:3300/0 conn(0x7f77c00af790 msgr2=0x7f77c0072b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:08.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.368+0000 7f77c4c2d640 1 --2- 172.21.15.5:0/3838620175 >> v2:172.21.15.80:3300/0 conn(0x7f77c00af790 0x7f77c0072b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:08.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.368+0000 7f77c4c2d640 1 -- 172.21.15.5:0/3838620175 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c006d4b0 con 0x7f77c00b26a0 2024-03-18T15:34:08.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.368+0000 7f77c4c2d640 1 -- 172.21.15.5:0/3838620175 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3837549003 0 0) 0x7f77b8001a00 con 0x7f77c00b26a0 2024-03-18T15:34:08.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.368+0000 7f77c4c2d640 1 -- 172.21.15.5:0/3838620175 >> v1:172.21.15.5:6789/0 conn(0x7f77c00b26a0 legacy=0x7f77c00b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:08.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.368+0000 7f77c4c2d640 1 --2- 172.21.15.5:0/3838620175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77ac003aa0 0x7f77ac003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:08.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.368+0000 7f77c4c2d640 1 --2- 172.21.15.5:0/3838620175 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f77c006d070 0x7f77ac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:08.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.368+0000 7f77c4c2d640 1 --2- 172.21.15.5:0/3838620175 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77ac004d50 0x7f77ac0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:08.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.368+0000 7f77c4c2d640 1 -- 172.21.15.5:0/3838620175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f77c00b4a20 con 0x7f77ac003aa0 2024-03-18T15:34:08.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.368+0000 7f77c4c2d640 1 -- 172.21.15.5:0/3838620175 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f77c00b4d00 con 0x7f77ac004d50 2024-03-18T15:34:08.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.368+0000 7f77c4c2d640 1 -- 172.21.15.5:0/3838620175 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f77ac00baa0 con 0x7f77c006d070 2024-03-18T15:34:08.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.368+0000 7f77c5c2f640 1 --2- 172.21.15.5:0/3838620175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77ac003aa0 0x7f77ac003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:08.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.368+0000 7f77c542e640 1 --2- 172.21.15.5:0/3838620175 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f77c006d070 0x7f77ac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:08.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.368+0000 7f77c5c2f640 1 -- 172.21.15.5:0/3838620175 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f77c006d070 msgr2=0x7f77ac004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:08.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.368+0000 7f77c5c2f640 1 --2- 172.21.15.5:0/3838620175 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f77c006d070 0x7f77ac004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:08.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.368+0000 7f77c6430640 1 --2- 172.21.15.5:0/3838620175 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77ac004d50 0x7f77ac0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:08.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.368+0000 7f77c5c2f640 1 -- 172.21.15.5:0/3838620175 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77ac004d50 msgr2=0x7f77ac0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:08.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.368+0000 7f77c5c2f640 1 --2- 172.21.15.5:0/3838620175 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77ac004d50 0x7f77ac0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:08.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.368+0000 7f77c5c2f640 1 -- 172.21.15.5:0/3838620175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77ac003530 con 0x7f77ac003aa0 2024-03-18T15:34:08.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.368+0000 7f77c6430640 1 --2- 172.21.15.5:0/3838620175 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77ac004d50 0x7f77ac0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:08.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.368+0000 7f77c5c2f640 1 --2- 172.21.15.5:0/3838620175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77ac003aa0 0x7f77ac003ec0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f77bc009ae0 tx=0x7f77bc01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34d6c424ac74198c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:08.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.368+0000 7f77c4c2d640 1 -- 172.21.15.5:0/3838620175 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77bc0096b0 con 0x7f77ac003aa0 2024-03-18T15:34:08.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.368+0000 7f77c4c2d640 1 -- 172.21.15.5:0/3838620175 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77bc0a64e0 con 0x7f77ac003aa0 2024-03-18T15:34:08.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.368+0000 7f77c4c2d640 1 -- 172.21.15.5:0/3838620175 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77bc0a68d0 con 0x7f77ac003aa0 2024-03-18T15:34:08.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.368+0000 7f77c7e93640 1 -- 172.21.15.5:0/3838620175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77ac003aa0 msgr2=0x7f77ac003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:08.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.368+0000 7f77c7e93640 1 --2- 172.21.15.5:0/3838620175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77ac003aa0 0x7f77ac003ec0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f77bc009ae0 tx=0x7f77bc01c9f0 comp rx=0 tx=0).stop 2024-03-18T15:34:08.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.368+0000 7f77c6430640 1 -- 172.21.15.5:0/3838620175 reap_dead start 2024-03-18T15:34:08.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.368+0000 7f77c7e93640 1 -- 172.21.15.5:0/3838620175 shutdown_connections 2024-03-18T15:34:08.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.368+0000 7f77c7e93640 1 -- 172.21.15.5:0/3838620175 >> 172.21.15.5:0/3838620175 conn(0x7f77c00b5670 msgr2=0x7f77c00b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:08.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.368+0000 7f77c7e93640 1 -- 172.21.15.5:0/3838620175 shutdown_connections 2024-03-18T15:34:08.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.368+0000 7f77c7e93640 1 -- 172.21.15.5:0/3838620175 wait complete. 2024-03-18T15:34:08.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.372+0000 7f77c7e93640 1 Processor -- start 2024-03-18T15:34:08.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.372+0000 7f77c7e93640 1 -- start start 2024-03-18T15:34:08.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.372+0000 7f77c7e93640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f77ac003aa0 0x7f77c0144970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:08.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.372+0000 7f77c7e93640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77c006fb30 0x7f77c006ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:08.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.372+0000 7f77c7e93640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77c013afe0 0x7f77c013b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:08.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.372+0000 7f77c7e93640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f77c014f690 con 0x7f77c013afe0 2024-03-18T15:34:08.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.372+0000 7f77c7e93640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f77c014f510 con 0x7f77c006fb30 2024-03-18T15:34:08.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.372+0000 7f77c7e93640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f77c014f810 con 0x7f77ac003aa0 2024-03-18T15:34:08.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.372+0000 7f77c6430640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77c013afe0 0x7f77c013b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:08.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.372+0000 7f77c6430640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77c013afe0 0x7f77c013b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50218/0 (socket says 172.21.15.5:50218) 2024-03-18T15:34:08.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.372+0000 7f77c6430640 1 -- 172.21.15.5:0/3873139856 learned_addr learned my addr 172.21.15.5:0/3873139856 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:08.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.372+0000 7f77c5c2f640 1 --2- 172.21.15.5:0/3873139856 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f77ac003aa0 0x7f77c0144970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:08.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.372+0000 7f77c542e640 1 --2- 172.21.15.5:0/3873139856 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77c006fb30 0x7f77c006ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:08.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.372+0000 7f77c6430640 1 -- 172.21.15.5:0/3873139856 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f77ac003aa0 msgr2=0x7f77c0144970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:08.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.372+0000 7f77c6430640 1 --2- 172.21.15.5:0/3873139856 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f77ac003aa0 0x7f77c0144970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:08.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.372+0000 7f77c6430640 1 -- 172.21.15.5:0/3873139856 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77c006fb30 msgr2=0x7f77c006ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:08.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.372+0000 7f77c6430640 1 --2- 172.21.15.5:0/3873139856 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77c006fb30 0x7f77c006ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:08.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.372+0000 7f77c6430640 1 -- 172.21.15.5:0/3873139856 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c00704f0 con 0x7f77c013afe0 2024-03-18T15:34:08.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.372+0000 7f77c6430640 1 --2- 172.21.15.5:0/3873139856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77c013afe0 0x7f77c013b460 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f77c00b9bf0 tx=0x7f77b800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:08.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.372+0000 7f77b67fc640 1 -- 172.21.15.5:0/3873139856 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b8004d70 con 0x7f77c013afe0 2024-03-18T15:34:08.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.372+0000 7f77b67fc640 1 -- 172.21.15.5:0/3873139856 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77b800bb20 con 0x7f77c013afe0 2024-03-18T15:34:08.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.372+0000 7f77c7e93640 1 -- 172.21.15.5:0/3873139856 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77c0071c90 con 0x7f77c013afe0 2024-03-18T15:34:08.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.372+0000 7f77b67fc640 1 -- 172.21.15.5:0/3873139856 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b8007110 con 0x7f77c013afe0 2024-03-18T15:34:08.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.372+0000 7f77c7e93640 1 -- 172.21.15.5:0/3873139856 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77c0143b50 con 0x7f77c013afe0 2024-03-18T15:34:08.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.372+0000 7f77c7e93640 1 -- 172.21.15.5:0/3873139856 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77c004f550 con 0x7f77c013afe0 2024-03-18T15:34:08.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.372+0000 7f77b67fc640 1 -- 172.21.15.5:0/3873139856 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f77b801b020 con 0x7f77c013afe0 2024-03-18T15:34:08.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.376+0000 7f77b67fc640 1 --2- 172.21.15.5:0/3873139856 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f77a4067660 0x7f77a4069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:08.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.376+0000 7f77c5c2f640 1 --2- 172.21.15.5:0/3873139856 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f77a4067660 0x7f77a4069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:08.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.376+0000 7f77b67fc640 1 -- 172.21.15.5:0/3873139856 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f77b8085960 con 0x7f77c013afe0 2024-03-18T15:34:08.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.376+0000 7f77c5c2f640 1 --2- 172.21.15.5:0/3873139856 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f77a4067660 0x7f77a4069b20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f77bc0af000 tx=0x7f77bc0046d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:08.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.380+0000 7f77b67fc640 1 -- 172.21.15.5:0/3873139856 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f77c00704f0 con 0x7f77c013afe0 2024-03-18T15:34:08.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.540+0000 7f77c7e93640 1 -- 172.21.15.5:0/3873139856 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f77c00713f0 con 0x7f77c013afe0 2024-03-18T15:34:08.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.540+0000 7f77b67fc640 1 -- 172.21.15.5:0/3873139856 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f77b80565d0 con 0x7f77c013afe0 2024-03-18T15:34:08.545 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-03-18T15:34:08.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.544+0000 7f77c7e93640 1 -- 172.21.15.5:0/3873139856 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f77a4067660 msgr2=0x7f77a4069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:08.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.544+0000 7f77c7e93640 1 --2- 172.21.15.5:0/3873139856 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f77a4067660 0x7f77a4069b20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f77bc0af000 tx=0x7f77bc0046d0 comp rx=0 tx=0).stop 2024-03-18T15:34:08.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.544+0000 7f77c7e93640 1 -- 172.21.15.5:0/3873139856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77c013afe0 msgr2=0x7f77c013b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:08.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.544+0000 7f77c7e93640 1 --2- 172.21.15.5:0/3873139856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77c013afe0 0x7f77c013b460 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f77c00b9bf0 tx=0x7f77b800b4a0 comp rx=0 tx=0).stop 2024-03-18T15:34:08.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.544+0000 7f77c7e93640 1 -- 172.21.15.5:0/3873139856 shutdown_connections 2024-03-18T15:34:08.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.544+0000 7f77c7e93640 1 --2- 172.21.15.5:0/3873139856 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f77a4067660 0x7f77a4069b20 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:08.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.544+0000 7f77c7e93640 1 --2- 172.21.15.5:0/3873139856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77c013afe0 0x7f77c013b460 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:08.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.544+0000 7f77c7e93640 1 --2- 172.21.15.5:0/3873139856 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77c006fb30 0x7f77c006ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:08.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.544+0000 7f77c7e93640 1 --2- 172.21.15.5:0/3873139856 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f77ac003aa0 0x7f77c0144970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:08.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.544+0000 7f77c7e93640 1 -- 172.21.15.5:0/3873139856 >> 172.21.15.5:0/3873139856 conn(0x7f77c00b5670 msgr2=0x7f77c014d360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:08.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.544+0000 7f77c7e93640 1 -- 172.21.15.5:0/3873139856 shutdown_connections 2024-03-18T15:34:08.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.544+0000 7f77c7e93640 1 -- 172.21.15.5:0/3873139856 wait complete. 2024-03-18T15:34:08.564 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-18T15:34:08.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.688+0000 7f359ab23640 1 Processor -- start 2024-03-18T15:34:08.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.688+0000 7f359ab23640 1 -- start start 2024-03-18T15:34:08.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.688+0000 7f359ab23640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f359413f260 0x7f359413f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:08.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.688+0000 7f359ab23640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f359414dc70 0x7f3594150060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:08.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.688+0000 7f359ab23640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35940b2ab0 con 0x7f359406d2d0 2024-03-18T15:34:08.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.688+0000 7f359ab23640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f35940728c0 con 0x7f359413f260 2024-03-18T15:34:08.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.688+0000 7f359ab23640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f35940b0ea0 con 0x7f359414dc70 2024-03-18T15:34:08.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.688+0000 7f3593fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f359413f260 0x7f359413f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:08.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.688+0000 7f35988bf640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f359406d2d0 0x7f35940b23a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60156/0 (socket says 172.21.15.5:60156) 2024-03-18T15:34:08.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.688+0000 7f35988bf640 1 -- 172.21.15.5:0/2504251922 learned_addr learned my addr 172.21.15.5:0/2504251922 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:08.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.688+0000 7f35990c0640 1 --2- 172.21.15.5:0/2504251922 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f359414dc70 0x7f3594150060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:08.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.688+0000 7f3593fff640 1 -- 172.21.15.5:0/2504251922 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f359414dc70 msgr2=0x7f3594150060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:08.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.688+0000 7f3593fff640 1 --2- 172.21.15.5:0/2504251922 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f359414dc70 0x7f3594150060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:08.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.688+0000 7f3593fff640 1 -- 172.21.15.5:0/2504251922 >> v1:172.21.15.5:6789/0 conn(0x7f359406d2d0 legacy=0x7f35940b23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:08.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.688+0000 7f35937fe640 1 -- 172.21.15.5:0/2504251922 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1584369043 0 0) 0x7f35940b2ab0 con 0x7f359406d2d0 2024-03-18T15:34:08.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.688+0000 7f3593fff640 1 -- 172.21.15.5:0/2504251922 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f359406d6b0 con 0x7f359413f260 2024-03-18T15:34:08.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.692+0000 7f3593fff640 1 --2- 172.21.15.5:0/2504251922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f359413f260 0x7f359413f6a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f35800097c0 tx=0x7f358001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34cbd8c812dbbb44 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:08.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.692+0000 7f35937fe640 1 -- 172.21.15.5:0/2504251922 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35800b0070 con 0x7f359413f260 2024-03-18T15:34:08.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.692+0000 7f35937fe640 1 -- 172.21.15.5:0/2504251922 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35800ab600 con 0x7f359413f260 2024-03-18T15:34:08.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.692+0000 7f35937fe640 1 -- 172.21.15.5:0/2504251922 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35800ab9d0 con 0x7f359413f260 2024-03-18T15:34:08.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.692+0000 7f359ab23640 1 -- 172.21.15.5:0/2504251922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f359413f260 msgr2=0x7f359413f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:08.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.692+0000 7f359ab23640 1 --2- 172.21.15.5:0/2504251922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f359413f260 0x7f359413f6a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f35800097c0 tx=0x7f358001c410 comp rx=0 tx=0).stop 2024-03-18T15:34:08.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.692+0000 7f359ab23640 1 -- 172.21.15.5:0/2504251922 shutdown_connections 2024-03-18T15:34:08.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.692+0000 7f359ab23640 1 --2- 172.21.15.5:0/2504251922 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f359414dc70 0x7f3594150060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:08.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.692+0000 7f359ab23640 1 --2- 172.21.15.5:0/2504251922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f359413f260 0x7f359413f6a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:08.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.692+0000 7f359ab23640 1 -- 172.21.15.5:0/2504251922 >> 172.21.15.5:0/2504251922 conn(0x7f35940b5670 msgr2=0x7f35940b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:08.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.692+0000 7f359ab23640 1 -- 172.21.15.5:0/2504251922 shutdown_connections 2024-03-18T15:34:08.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.692+0000 7f359ab23640 1 -- 172.21.15.5:0/2504251922 wait complete. 2024-03-18T15:34:08.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.692+0000 7f359ab23640 1 Processor -- start 2024-03-18T15:34:08.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.692+0000 7f359ab23640 1 -- start start 2024-03-18T15:34:08.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.696+0000 7f359ab23640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f359406d2d0 0x7f35941e1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:08.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.696+0000 7f359ab23640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f359413f260 0x7f35941388a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:08.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.696+0000 7f359ab23640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f359414dc70 0x7f3594136340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:08.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.696+0000 7f359ab23640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f35940b0960 con 0x7f359414dc70 2024-03-18T15:34:08.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.696+0000 7f359ab23640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f35940b07e0 con 0x7f359406d2d0 2024-03-18T15:34:08.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.696+0000 7f359ab23640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f3594072d50 con 0x7f359413f260 2024-03-18T15:34:08.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.696+0000 7f35990c0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f359414dc70 0x7f3594136340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:08.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.696+0000 7f35990c0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f359414dc70 0x7f3594136340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50224/0 (socket says 172.21.15.5:50224) 2024-03-18T15:34:08.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.696+0000 7f35990c0640 1 -- 172.21.15.5:0/99345581 learned_addr learned my addr 172.21.15.5:0/99345581 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:08.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.696+0000 7f35988bf640 1 --2- 172.21.15.5:0/99345581 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f359406d2d0 0x7f35941e1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:08.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.696+0000 7f3593fff640 1 --2- 172.21.15.5:0/99345581 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f359413f260 0x7f35941388a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:08.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.696+0000 7f35990c0640 1 -- 172.21.15.5:0/99345581 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f359413f260 msgr2=0x7f35941388a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:08.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.696+0000 7f35990c0640 1 --2- 172.21.15.5:0/99345581 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f359413f260 0x7f35941388a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:08.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.696+0000 7f35990c0640 1 -- 172.21.15.5:0/99345581 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f359406d2d0 msgr2=0x7f35941e1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:08.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.696+0000 7f35990c0640 1 --2- 172.21.15.5:0/99345581 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f359406d2d0 0x7f35941e1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:08.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.696+0000 7f35990c0640 1 -- 172.21.15.5:0/99345581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35941380a0 con 0x7f359414dc70 2024-03-18T15:34:08.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.696+0000 7f3593fff640 1 --2- 172.21.15.5:0/99345581 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f359413f260 0x7f35941388a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:34:08.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.696+0000 7f35988bf640 1 --2- 172.21.15.5:0/99345581 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f359406d2d0 0x7f35941e1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:34:08.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.696+0000 7f35990c0640 1 --2- 172.21.15.5:0/99345581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f359414dc70 0x7f3594136340 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f358c00ef90 tx=0x7f358c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:08.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.696+0000 7f35917fa640 1 -- 172.21.15.5:0/99345581 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c018020 con 0x7f359414dc70 2024-03-18T15:34:08.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.696+0000 7f35917fa640 1 -- 172.21.15.5:0/99345581 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f358c014480 con 0x7f359414dc70 2024-03-18T15:34:08.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.696+0000 7f359ab23640 1 -- 172.21.15.5:0/99345581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3594133ec0 con 0x7f359414dc70 2024-03-18T15:34:08.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.696+0000 7f35917fa640 1 -- 172.21.15.5:0/99345581 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358c009e60 con 0x7f359414dc70 2024-03-18T15:34:08.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.696+0000 7f359ab23640 1 -- 172.21.15.5:0/99345581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3594136880 con 0x7f359414dc70 2024-03-18T15:34:08.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.696+0000 7f359ab23640 1 -- 172.21.15.5:0/99345581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3594137e90 con 0x7f359414dc70 2024-03-18T15:34:08.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.696+0000 7f35917fa640 1 -- 172.21.15.5:0/99345581 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f358c01b020 con 0x7f359414dc70 2024-03-18T15:34:08.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.696+0000 7f35917fa640 1 --2- 172.21.15.5:0/99345581 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3578067690 0x7f3578069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:08.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.696+0000 7f35917fa640 1 -- 172.21.15.5:0/99345581 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f358c086530 con 0x7f359414dc70 2024-03-18T15:34:08.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.696+0000 7f35988bf640 1 --2- 172.21.15.5:0/99345581 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3578067690 0x7f3578069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:08.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.700+0000 7f35988bf640 1 --2- 172.21.15.5:0/99345581 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3578067690 0x7f3578069b50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3588007fd0 tx=0x7f3588007c20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:08.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.700+0000 7f35917fa640 1 -- 172.21.15.5:0/99345581 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f358c022030 con 0x7f359414dc70 2024-03-18T15:34:08.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.876+0000 7f359ab23640 1 -- 172.21.15.5:0/99345581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f359413d8e0 con 0x7f359414dc70 2024-03-18T15:34:08.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.876+0000 7f35917fa640 1 -- 172.21.15.5:0/99345581 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7f358c010040 con 0x7f359414dc70 2024-03-18T15:34:08.880 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2024-03-18T15:34:08.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.880+0000 7f359ab23640 1 -- 172.21.15.5:0/99345581 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3578067690 msgr2=0x7f3578069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:08.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.880+0000 7f359ab23640 1 --2- 172.21.15.5:0/99345581 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3578067690 0x7f3578069b50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3588007fd0 tx=0x7f3588007c20 comp rx=0 tx=0).stop 2024-03-18T15:34:08.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.880+0000 7f359ab23640 1 -- 172.21.15.5:0/99345581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f359414dc70 msgr2=0x7f3594136340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:08.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.880+0000 7f359ab23640 1 --2- 172.21.15.5:0/99345581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f359414dc70 0x7f3594136340 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f358c00ef90 tx=0x7f358c00c490 comp rx=0 tx=0).stop 2024-03-18T15:34:08.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.880+0000 7f359ab23640 1 -- 172.21.15.5:0/99345581 shutdown_connections 2024-03-18T15:34:08.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.880+0000 7f359ab23640 1 --2- 172.21.15.5:0/99345581 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3578067690 0x7f3578069b50 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:08.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.880+0000 7f359ab23640 1 --2- 172.21.15.5:0/99345581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f359414dc70 0x7f3594136340 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:08.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.880+0000 7f359ab23640 1 --2- 172.21.15.5:0/99345581 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f359413f260 0x7f35941388a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:08.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.880+0000 7f359ab23640 1 --2- 172.21.15.5:0/99345581 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f359406d2d0 0x7f35941e1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:08.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.880+0000 7f359ab23640 1 -- 172.21.15.5:0/99345581 >> 172.21.15.5:0/99345581 conn(0x7f35940b5670 msgr2=0x7f359413ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:08.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.880+0000 7f359ab23640 1 -- 172.21.15.5:0/99345581 shutdown_connections 2024-03-18T15:34:08.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:08.880+0000 7f359ab23640 1 -- 172.21.15.5:0/99345581 wait complete. 2024-03-18T15:34:08.900 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi005.front.sepia.ceph.com... 2024-03-18T15:34:08.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:34:08.900 DEBUG:teuthology.orchestra.run.smithi005:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-18T15:34:08.925 INFO:teuthology.orchestra.run:waiting for 300 2024-03-18T15:34:08.925 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-03-18T15:34:08.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:34:08.926 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-18T15:34:08.985 INFO:tasks.cephfs.cephfs_test_case:[] 2024-03-18T15:34:08.985 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:34:09.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.156+0000 7f8a652b0640 1 Processor -- start 2024-03-18T15:34:09.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.156+0000 7f8a652b0640 1 -- start start 2024-03-18T15:34:09.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.156+0000 7f8a652b0640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f8a600af790 0x7f8a60072b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:09.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.156+0000 7f8a652b0640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a600b26a0 0x7f8a600b2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:09.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.156+0000 7f8a652b0640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a600b3070 con 0x7f8a6006d070 2024-03-18T15:34:09.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.156+0000 7f8a652b0640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f8a600afb70 con 0x7f8a600af790 2024-03-18T15:34:09.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.156+0000 7f8a652b0640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f8a600b4a20 con 0x7f8a600b26a0 2024-03-18T15:34:09.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.160+0000 7f8a5e7fc640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f8a6006d070 0x7f8a60147780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60164/0 (socket says 172.21.15.5:60164) 2024-03-18T15:34:09.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.160+0000 7f8a5e7fc640 1 -- 172.21.15.5:0/1106892077 learned_addr learned my addr 172.21.15.5:0/1106892077 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:09.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.160+0000 7f8a5f7fe640 1 --2- 172.21.15.5:0/1106892077 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a600b26a0 0x7f8a600b2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:09.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.160+0000 7f8a5effd640 1 --2- 172.21.15.5:0/1106892077 >> v2:172.21.15.80:3300/0 conn(0x7f8a600af790 0x7f8a60072b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:09.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.160+0000 7f8a5dffb640 1 -- 172.21.15.5:0/1106892077 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1350546030 0 0) 0x7f8a600b3070 con 0x7f8a6006d070 2024-03-18T15:34:09.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.160+0000 7f8a5dffb640 1 -- 172.21.15.5:0/1106892077 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8a48003530 con 0x7f8a6006d070 2024-03-18T15:34:09.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.160+0000 7f8a5dffb640 1 -- 172.21.15.5:0/1106892077 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 911911966 0 0) 0x7f8a48003530 con 0x7f8a6006d070 2024-03-18T15:34:09.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.160+0000 7f8a5dffb640 1 -- 172.21.15.5:0/1106892077 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a600b26a0 msgr2=0x7f8a600b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:09.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.160+0000 7f8a5dffb640 1 --2- 172.21.15.5:0/1106892077 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a600b26a0 0x7f8a600b2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:09.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.160+0000 7f8a5dffb640 1 -- 172.21.15.5:0/1106892077 >> v2:172.21.15.80:3300/0 conn(0x7f8a600af790 msgr2=0x7f8a60072b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:09.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.160+0000 7f8a5dffb640 1 --2- 172.21.15.5:0/1106892077 >> v2:172.21.15.80:3300/0 conn(0x7f8a600af790 0x7f8a60072b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:09.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.160+0000 7f8a5dffb640 1 -- 172.21.15.5:0/1106892077 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a600b4d00 con 0x7f8a6006d070 2024-03-18T15:34:09.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.160+0000 7f8a5dffb640 1 -- 172.21.15.5:0/1106892077 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3837549003 0 0) 0x7f8a54001680 con 0x7f8a6006d070 2024-03-18T15:34:09.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.160+0000 7f8a5dffb640 1 -- 172.21.15.5:0/1106892077 >> v1:172.21.15.5:6789/0 conn(0x7f8a6006d070 legacy=0x7f8a60147780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:09.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.160+0000 7f8a5dffb640 1 --2- 172.21.15.5:0/1106892077 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a600b26a0 0x7f8a48003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:09.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.160+0000 7f8a5dffb640 1 --2- 172.21.15.5:0/1106892077 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a48004020 0x7f8a48004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:09.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.160+0000 7f8a5dffb640 1 --2- 172.21.15.5:0/1106892077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a48004d50 0x7f8a480051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:09.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.160+0000 7f8a5dffb640 1 -- 172.21.15.5:0/1106892077 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8a600afb70 con 0x7f8a48004d50 2024-03-18T15:34:09.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.160+0000 7f8a5dffb640 1 -- 172.21.15.5:0/1106892077 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8a600b4a20 con 0x7f8a48004020 2024-03-18T15:34:09.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.160+0000 7f8a5dffb640 1 -- 172.21.15.5:0/1106892077 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f8a4800bb00 con 0x7f8a600b26a0 2024-03-18T15:34:09.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.160+0000 7f8a5f7fe640 1 --2- 172.21.15.5:0/1106892077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a48004d50 0x7f8a480051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:09.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.160+0000 7f8a5effd640 1 --2- 172.21.15.5:0/1106892077 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a600b26a0 0x7f8a48003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:09.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.160+0000 7f8a5e7fc640 1 --2- 172.21.15.5:0/1106892077 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a48004020 0x7f8a48004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:09.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.160+0000 7f8a5f7fe640 1 -- 172.21.15.5:0/1106892077 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a600b26a0 msgr2=0x7f8a48003aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:09.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.160+0000 7f8a5f7fe640 1 --2- 172.21.15.5:0/1106892077 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a600b26a0 0x7f8a48003aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:09.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.160+0000 7f8a5f7fe640 1 -- 172.21.15.5:0/1106892077 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a48004020 msgr2=0x7f8a48004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:09.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.160+0000 7f8a5f7fe640 1 --2- 172.21.15.5:0/1106892077 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a48004020 0x7f8a48004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:09.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.160+0000 7f8a5f7fe640 1 -- 172.21.15.5:0/1106892077 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a48003530 con 0x7f8a48004d50 2024-03-18T15:34:09.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.160+0000 7f8a5e7fc640 1 --2- 172.21.15.5:0/1106892077 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a48004020 0x7f8a48004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:09.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.160+0000 7f8a5f7fe640 1 --2- 172.21.15.5:0/1106892077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a48004d50 0x7f8a480051d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8a58009fc0 tx=0x7f8a5801cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cfa444471482f45 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:09.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.160+0000 7f8a5dffb640 1 -- 172.21.15.5:0/1106892077 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a580b2070 con 0x7f8a48004d50 2024-03-18T15:34:09.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.160+0000 7f8a5dffb640 1 -- 172.21.15.5:0/1106892077 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a580a5c90 con 0x7f8a48004d50 2024-03-18T15:34:09.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.160+0000 7f8a5dffb640 1 -- 172.21.15.5:0/1106892077 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a58009c10 con 0x7f8a48004d50 2024-03-18T15:34:09.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.164+0000 7f8a652b0640 1 -- 172.21.15.5:0/1106892077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a48004d50 msgr2=0x7f8a480051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:09.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.164+0000 7f8a652b0640 1 --2- 172.21.15.5:0/1106892077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a48004d50 0x7f8a480051d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8a58009fc0 tx=0x7f8a5801cc80 comp rx=0 tx=0).stop 2024-03-18T15:34:09.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.164+0000 7f8a5f7fe640 1 -- 172.21.15.5:0/1106892077 reap_dead start 2024-03-18T15:34:09.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.164+0000 7f8a652b0640 1 -- 172.21.15.5:0/1106892077 shutdown_connections 2024-03-18T15:34:09.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.164+0000 7f8a652b0640 1 -- 172.21.15.5:0/1106892077 >> 172.21.15.5:0/1106892077 conn(0x7f8a600b5670 msgr2=0x7f8a600b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:09.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.164+0000 7f8a652b0640 1 -- 172.21.15.5:0/1106892077 shutdown_connections 2024-03-18T15:34:09.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.164+0000 7f8a652b0640 1 -- 172.21.15.5:0/1106892077 wait complete. 2024-03-18T15:34:09.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.164+0000 7f8a652b0640 1 Processor -- start 2024-03-18T15:34:09.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.164+0000 7f8a652b0640 1 -- start start 2024-03-18T15:34:09.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.164+0000 7f8a652b0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a6013f2f0 0x7f8a60140240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.164+0000 7f8a652b0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a60071f00 0x7f8a60141170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.164+0000 7f8a652b0640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a601dfe30 0x7f8a601e02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.164+0000 7f8a652b0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8a600b5220 con 0x7f8a6013f2f0 2024-03-18T15:34:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.164+0000 7f8a652b0640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8a600b2b30 con 0x7f8a60071f00 2024-03-18T15:34:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.164+0000 7f8a652b0640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f8a60071a70 con 0x7f8a601dfe30 2024-03-18T15:34:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.164+0000 7f8a5effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a6013f2f0 0x7f8a60140240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.164+0000 7f8a5effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a6013f2f0 0x7f8a60140240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50248/0 (socket says 172.21.15.5:50248) 2024-03-18T15:34:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.164+0000 7f8a5effd640 1 -- 172.21.15.5:0/4220887459 learned_addr learned my addr 172.21.15.5:0/4220887459 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.164+0000 7f8a5e7fc640 1 --2- 172.21.15.5:0/4220887459 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a60071f00 0x7f8a60141170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.164+0000 7f8a5f7fe640 1 --2- 172.21.15.5:0/4220887459 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a601dfe30 0x7f8a601e02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.164+0000 7f8a5effd640 1 -- 172.21.15.5:0/4220887459 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a601dfe30 msgr2=0x7f8a601e02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.164+0000 7f8a5effd640 1 --2- 172.21.15.5:0/4220887459 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a601dfe30 0x7f8a601e02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.164+0000 7f8a5effd640 1 -- 172.21.15.5:0/4220887459 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a60071f00 msgr2=0x7f8a60141170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:09.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.164+0000 7f8a5effd640 1 --2- 172.21.15.5:0/4220887459 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a60071f00 0x7f8a60141170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:09.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.164+0000 7f8a5effd640 1 -- 172.21.15.5:0/4220887459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a60140a00 con 0x7f8a6013f2f0 2024-03-18T15:34:09.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.164+0000 7f8a5effd640 1 --2- 172.21.15.5:0/4220887459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a6013f2f0 0x7f8a60140240 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8a50010930 tx=0x7f8a50010df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:09.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.164+0000 7f8a477fe640 1 -- 172.21.15.5:0/4220887459 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5000f040 con 0x7f8a6013f2f0 2024-03-18T15:34:09.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.164+0000 7f8a477fe640 1 -- 172.21.15.5:0/4220887459 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a5000ec20 con 0x7f8a6013f2f0 2024-03-18T15:34:09.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.164+0000 7f8a652b0640 1 -- 172.21.15.5:0/4220887459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a60071700 con 0x7f8a6013f2f0 2024-03-18T15:34:09.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.164+0000 7f8a477fe640 1 -- 172.21.15.5:0/4220887459 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5001c400 con 0x7f8a6013f2f0 2024-03-18T15:34:09.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.164+0000 7f8a652b0640 1 -- 172.21.15.5:0/4220887459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a6013eb40 con 0x7f8a6013f2f0 2024-03-18T15:34:09.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.164+0000 7f8a652b0640 1 -- 172.21.15.5:0/4220887459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a6004f550 con 0x7f8a6013f2f0 2024-03-18T15:34:09.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.168+0000 7f8a477fe640 1 -- 172.21.15.5:0/4220887459 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f8a5001c5a0 con 0x7f8a6013f2f0 2024-03-18T15:34:09.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.168+0000 7f8a477fe640 1 --2- 172.21.15.5:0/4220887459 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f8a4c067660 0x7f8a4c069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:09.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.168+0000 7f8a477fe640 1 -- 172.21.15.5:0/4220887459 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8a50086220 con 0x7f8a6013f2f0 2024-03-18T15:34:09.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.168+0000 7f8a5e7fc640 1 --2- 172.21.15.5:0/4220887459 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f8a4c067660 0x7f8a4c069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:09.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.168+0000 7f8a5e7fc640 1 --2- 172.21.15.5:0/4220887459 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f8a4c067660 0x7f8a4c069b20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8a6013e3f0 tx=0x7f8a54009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:09.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.172+0000 7f8a477fe640 1 -- 172.21.15.5:0/4220887459 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f8a50018030 con 0x7f8a6013f2f0 2024-03-18T15:34:09.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.368+0000 7f8a652b0640 1 -- 172.21.15.5:0/4220887459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a600b53a0 con 0x7f8a6013f2f0 2024-03-18T15:34:09.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.368+0000 7f8a477fe640 1 -- 172.21.15.5:0/4220887459 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+3521 (secure 0 0 0) 0x7f8a500570d0 con 0x7f8a6013f2f0 2024-03-18T15:34:09.372 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:34:09.372 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/629589896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":629589896},{"type":"v1","addr":"172.21.15.5:6835","nonce":629589896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-18T15:33:42.338926+0000","modified":"2024-03-18T15:33:45.376646+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6833/1236603350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":1236603350},{"type":"v1","addr":"172.21.15.80:6833","nonce":1236603350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4272,"qdb_cluster":[4272]},"id":1}]} 2024-03-18T15:34:09.373 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 8 2024-03-18T15:34:09.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.372+0000 7f8a652b0640 1 -- 172.21.15.5:0/4220887459 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f8a4c067660 msgr2=0x7f8a4c069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:09.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.372+0000 7f8a652b0640 1 --2- 172.21.15.5:0/4220887459 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f8a4c067660 0x7f8a4c069b20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8a6013e3f0 tx=0x7f8a54009040 comp rx=0 tx=0).stop 2024-03-18T15:34:09.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.372+0000 7f8a652b0640 1 -- 172.21.15.5:0/4220887459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a6013f2f0 msgr2=0x7f8a60140240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:09.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.372+0000 7f8a652b0640 1 --2- 172.21.15.5:0/4220887459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a6013f2f0 0x7f8a60140240 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8a50010930 tx=0x7f8a50010df0 comp rx=0 tx=0).stop 2024-03-18T15:34:09.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.372+0000 7f8a652b0640 1 -- 172.21.15.5:0/4220887459 shutdown_connections 2024-03-18T15:34:09.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.372+0000 7f8a652b0640 1 --2- 172.21.15.5:0/4220887459 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f8a4c067660 0x7f8a4c069b20 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:09.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.372+0000 7f8a652b0640 1 --2- 172.21.15.5:0/4220887459 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8a601dfe30 0x7f8a601e02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:09.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.372+0000 7f8a652b0640 1 --2- 172.21.15.5:0/4220887459 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a60071f00 0x7f8a60141170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:09.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.372+0000 7f8a652b0640 1 --2- 172.21.15.5:0/4220887459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a6013f2f0 0x7f8a60140240 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:09.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.372+0000 7f8a652b0640 1 -- 172.21.15.5:0/4220887459 >> 172.21.15.5:0/4220887459 conn(0x7f8a600b5670 msgr2=0x7f8a6006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:09.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.372+0000 7f8a652b0640 1 -- 172.21.15.5:0/4220887459 shutdown_connections 2024-03-18T15:34:09.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.372+0000 7f8a652b0640 1 -- 172.21.15.5:0/4220887459 wait complete. 2024-03-18T15:34:09.392 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:34:09.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.508+0000 7f5d00628640 1 Processor -- start 2024-03-18T15:34:09.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.508+0000 7f5d00628640 1 -- start start 2024-03-18T15:34:09.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.508+0000 7f5d00628640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5cf813f6a0 0x7f5cf81513a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:09.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.508+0000 7f5d00628640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cf80b2ab0 con 0x7f5cf813f260 2024-03-18T15:34:09.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.508+0000 7f5d00628640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cf813fb10 con 0x7f5cf806d2d0 2024-03-18T15:34:09.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.508+0000 7f5d00628640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f5cf80728c0 con 0x7f5cf813f6a0 2024-03-18T15:34:09.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.508+0000 7f5cfdbc3640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f5cf813f260 0x7f5cf814dc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60170/0 (socket says 172.21.15.5:60170) 2024-03-18T15:34:09.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.508+0000 7f5cfdbc3640 1 -- 172.21.15.5:0/1595835872 learned_addr learned my addr 172.21.15.5:0/1595835872 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:09.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.508+0000 7f5cfebc5640 1 --2- 172.21.15.5:0/1595835872 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5cf813f6a0 0x7f5cf81513a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:09.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.508+0000 7f5cfd3c2640 1 -- 172.21.15.5:0/1595835872 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2602554667 0 0) 0x7f5cf80b2ab0 con 0x7f5cf813f260 2024-03-18T15:34:09.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.508+0000 7f5cfd3c2640 1 -- 172.21.15.5:0/1595835872 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ce4003530 con 0x7f5cf813f260 2024-03-18T15:34:09.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.508+0000 7f5cfd3c2640 1 -- 172.21.15.5:0/1595835872 <== mon.1 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 984996427 0 0) 0x7f5cf813fb10 con 0x7f5cf806d2d0 2024-03-18T15:34:09.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.508+0000 7f5cfd3c2640 1 -- 172.21.15.5:0/1595835872 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5cf80b2ab0 con 0x7f5cf806d2d0 2024-03-18T15:34:09.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.508+0000 7f5cfd3c2640 1 -- 172.21.15.5:0/1595835872 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3620952864 0 0) 0x7f5ce4003530 con 0x7f5cf813f260 2024-03-18T15:34:09.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.508+0000 7f5cfd3c2640 1 -- 172.21.15.5:0/1595835872 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5cf813f6a0 msgr2=0x7f5cf81513a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:09.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.508+0000 7f5cfd3c2640 1 --2- 172.21.15.5:0/1595835872 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5cf813f6a0 0x7f5cf81513a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:09.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.508+0000 7f5cfd3c2640 1 -- 172.21.15.5:0/1595835872 >> v1:172.21.15.80:6789/0 conn(0x7f5cf806d2d0 legacy=0x7f5cf80b23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:09.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.508+0000 7f5cfd3c2640 1 -- 172.21.15.5:0/1595835872 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cf80b0ea0 con 0x7f5cf813f260 2024-03-18T15:34:09.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.508+0000 7f5cfd3c2640 1 -- 172.21.15.5:0/1595835872 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3837549003 0 0) 0x7f5cf4001680 con 0x7f5cf813f260 2024-03-18T15:34:09.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.508+0000 7f5cfd3c2640 1 -- 172.21.15.5:0/1595835872 >> v1:172.21.15.5:6789/0 conn(0x7f5cf813f260 legacy=0x7f5cf814dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:09.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.508+0000 7f5cfd3c2640 1 --2- 172.21.15.5:0/1595835872 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5cf813f6a0 0x7f5ce4003c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:09.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.508+0000 7f5cfd3c2640 1 --2- 172.21.15.5:0/1595835872 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ce40041e0 0x7f5ce4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:09.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.508+0000 7f5cfd3c2640 1 --2- 172.21.15.5:0/1595835872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ce4004d50 0x7f5ce40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:09.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.508+0000 7f5cfd3c2640 1 -- 172.21.15.5:0/1595835872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf80728c0 con 0x7f5ce4004d50 2024-03-18T15:34:09.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.508+0000 7f5cfd3c2640 1 -- 172.21.15.5:0/1595835872 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce400bc30 con 0x7f5ce40041e0 2024-03-18T15:34:09.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.508+0000 7f5cfd3c2640 1 -- 172.21.15.5:0/1595835872 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f5ce400be30 con 0x7f5cf813f6a0 2024-03-18T15:34:09.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.512+0000 7f5cfebc5640 1 --2- 172.21.15.5:0/1595835872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ce4004d50 0x7f5ce40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:09.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.512+0000 7f5cfdbc3640 1 --2- 172.21.15.5:0/1595835872 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ce40041e0 0x7f5ce4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:09.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.512+0000 7f5cfe3c4640 1 --2- 172.21.15.5:0/1595835872 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5cf813f6a0 0x7f5ce4003c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:09.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.512+0000 7f5cfebc5640 1 -- 172.21.15.5:0/1595835872 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5cf813f6a0 msgr2=0x7f5ce4003c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:09.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.512+0000 7f5cfebc5640 1 --2- 172.21.15.5:0/1595835872 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5cf813f6a0 0x7f5ce4003c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:09.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.512+0000 7f5cfebc5640 1 -- 172.21.15.5:0/1595835872 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ce40041e0 msgr2=0x7f5ce4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:09.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.512+0000 7f5cfebc5640 1 --2- 172.21.15.5:0/1595835872 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ce40041e0 0x7f5ce4004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:09.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.512+0000 7f5cfebc5640 1 -- 172.21.15.5:0/1595835872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ce4003530 con 0x7f5ce4004d50 2024-03-18T15:34:09.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.512+0000 7f5cfe3c4640 1 --2- 172.21.15.5:0/1595835872 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5cf813f6a0 0x7f5ce4003c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:09.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.512+0000 7f5cfebc5640 1 --2- 172.21.15.5:0/1595835872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ce4004d50 0x7f5ce40051d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0009fa0 tx=0x7f5cf001cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f978b8f6e4e0e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:09.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.512+0000 7f5cfd3c2640 1 -- 172.21.15.5:0/1595835872 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf00b2070 con 0x7f5ce4004d50 2024-03-18T15:34:09.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.512+0000 7f5cfd3c2640 1 -- 172.21.15.5:0/1595835872 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cf00a5d10 con 0x7f5ce4004d50 2024-03-18T15:34:09.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.512+0000 7f5cfd3c2640 1 -- 172.21.15.5:0/1595835872 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0009bf0 con 0x7f5ce4004d50 2024-03-18T15:34:09.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.512+0000 7f5d00628640 1 -- 172.21.15.5:0/1595835872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ce4004d50 msgr2=0x7f5ce40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:09.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.512+0000 7f5d00628640 1 --2- 172.21.15.5:0/1595835872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ce4004d50 0x7f5ce40051d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0009fa0 tx=0x7f5cf001cc40 comp rx=0 tx=0).stop 2024-03-18T15:34:09.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.512+0000 7f5cfebc5640 1 -- 172.21.15.5:0/1595835872 reap_dead start 2024-03-18T15:34:09.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.512+0000 7f5d00628640 1 -- 172.21.15.5:0/1595835872 shutdown_connections 2024-03-18T15:34:09.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.512+0000 7f5d00628640 1 -- 172.21.15.5:0/1595835872 >> 172.21.15.5:0/1595835872 conn(0x7f5cf80b5670 msgr2=0x7f5cf80b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:09.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.512+0000 7f5d00628640 1 -- 172.21.15.5:0/1595835872 shutdown_connections 2024-03-18T15:34:09.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.512+0000 7f5d00628640 1 -- 172.21.15.5:0/1595835872 wait complete. 2024-03-18T15:34:09.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.516+0000 7f5d00628640 1 Processor -- start 2024-03-18T15:34:09.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.516+0000 7f5d00628640 1 -- start start 2024-03-18T15:34:09.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.516+0000 7f5d00628640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cf8149110 0x7f5cf8146cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:09.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.516+0000 7f5d00628640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5cf81481a0 0x7f5cf8147230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:09.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.516+0000 7f5d00628640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5cf8132c30 0x7f5cf81330e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:09.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.516+0000 7f5d00628640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf80b1c30 con 0x7f5cf8149110 2024-03-18T15:34:09.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.516+0000 7f5d00628640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf80b0bc0 con 0x7f5cf81481a0 2024-03-18T15:34:09.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.516+0000 7f5d00628640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f5cf80b0070 con 0x7f5cf8132c30 2024-03-18T15:34:09.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.516+0000 7f5cfe3c4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cf8149110 0x7f5cf8146cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:09.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.516+0000 7f5cfe3c4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cf8149110 0x7f5cf8146cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50264/0 (socket says 172.21.15.5:50264) 2024-03-18T15:34:09.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.516+0000 7f5cfe3c4640 1 -- 172.21.15.5:0/1405479296 learned_addr learned my addr 172.21.15.5:0/1405479296 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:09.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.516+0000 7f5cfebc5640 1 --2- 172.21.15.5:0/1405479296 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5cf8132c30 0x7f5cf81330e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:09.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.516+0000 7f5cfdbc3640 1 --2- 172.21.15.5:0/1405479296 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5cf81481a0 0x7f5cf8147230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:09.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.516+0000 7f5cfe3c4640 1 -- 172.21.15.5:0/1405479296 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5cf8132c30 msgr2=0x7f5cf81330e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:09.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.516+0000 7f5cfe3c4640 1 --2- 172.21.15.5:0/1405479296 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5cf8132c30 0x7f5cf81330e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:09.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.516+0000 7f5cfe3c4640 1 -- 172.21.15.5:0/1405479296 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5cf81481a0 msgr2=0x7f5cf8147230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:09.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.516+0000 7f5cfe3c4640 1 --2- 172.21.15.5:0/1405479296 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5cf81481a0 0x7f5cf8147230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:09.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.516+0000 7f5cfe3c4640 1 -- 172.21.15.5:0/1405479296 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cf81486b0 con 0x7f5cf8149110 2024-03-18T15:34:09.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.516+0000 7f5cfdbc3640 1 --2- 172.21.15.5:0/1405479296 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5cf81481a0 0x7f5cf8147230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:09.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.516+0000 7f5cfe3c4640 1 --2- 172.21.15.5:0/1405479296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cf8149110 0x7f5cf8146cf0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5ce800f910 tx=0x7f5ce800fdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:09.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.516+0000 7f5ceeffd640 1 -- 172.21.15.5:0/1405479296 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce800d930 con 0x7f5cf8149110 2024-03-18T15:34:09.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.516+0000 7f5ceeffd640 1 -- 172.21.15.5:0/1405479296 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ce800dad0 con 0x7f5cf8149110 2024-03-18T15:34:09.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.516+0000 7f5ceeffd640 1 -- 172.21.15.5:0/1405479296 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce80067e0 con 0x7f5cf8149110 2024-03-18T15:34:09.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.516+0000 7f5d00628640 1 -- 172.21.15.5:0/1405479296 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cf81494f0 con 0x7f5cf8149110 2024-03-18T15:34:09.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.516+0000 7f5d00628640 1 -- 172.21.15.5:0/1405479296 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cf8145d20 con 0x7f5cf8149110 2024-03-18T15:34:09.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.516+0000 7f5d00628640 1 -- 172.21.15.5:0/1405479296 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cf804f550 con 0x7f5cf8149110 2024-03-18T15:34:09.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.516+0000 7f5ceeffd640 1 -- 172.21.15.5:0/1405479296 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f5ce8006980 con 0x7f5cf8149110 2024-03-18T15:34:09.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.516+0000 7f5ceeffd640 1 --2- 172.21.15.5:0/1405479296 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5cdc067690 0x7f5cdc069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:09.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.516+0000 7f5ceeffd640 1 -- 172.21.15.5:0/1405479296 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f5ce8088980 con 0x7f5cf8149110 2024-03-18T15:34:09.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.520+0000 7f5cfdbc3640 1 --2- 172.21.15.5:0/1405479296 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5cdc067690 0x7f5cdc069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:09.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.520+0000 7f5cfdbc3640 1 --2- 172.21.15.5:0/1405479296 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5cdc067690 0x7f5cdc069b50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4001ec0 tx=0x7f5cf4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:09.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.524+0000 7f5ceeffd640 1 -- 172.21.15.5:0/1405479296 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f5cf81486b0 con 0x7f5cf8149110 2024-03-18T15:34:09.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.688+0000 7f5d00628640 1 -- 172.21.15.5:0/1405479296 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5cf806ccc0 con 0x7f5cf8149110 2024-03-18T15:34:09.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.688+0000 7f5ceeffd640 1 -- 172.21.15.5:0/1405479296 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16449 (secure 0 0 0) 0x7f5ce8018020 con 0x7f5cf8149110 2024-03-18T15:34:09.692 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:34:09.692 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":21,"fsid":"ef54fe03-5aa7-4eb2-9b1e-66c9cc345b94","created":"2024-03-18T15:33:18.086614+0000","modified":"2024-03-18T15:33:43.984050+0000","last_up_change":"2024-03-18T15:33:29.229911+0000","last_in_change":"2024-03-18T15:33:20.323840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:28.963186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T15:33:40.533646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T15:33:41.323519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df1c754b-af59-4d16-8919-fbb6386b3c48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6804","nonce":976264754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6808","nonce":976264754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6815","nonce":976264754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6811","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6813","nonce":976264754}]},"public_addr":"172.21.15.5:6804/976264754","cluster_addr":"172.21.15.5:6808/976264754","heartbeat_back_addr":"172.21.15.5:6815/976264754","heartbeat_front_addr":"172.21.15.5:6813/976264754","state":["exists","up"]},{"osd":1,"uuid":"b9a08b5e-8af4-4144-bdba-d69ed1ac627a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6801","nonce":3901761126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6803","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6805","nonce":3901761126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6812","nonce":3901761126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6807","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6809","nonce":3901761126}]},"public_addr":"172.21.15.5:6801/3901761126","cluster_addr":"172.21.15.5:6805/3901761126","heartbeat_back_addr":"172.21.15.5:6812/3901761126","heartbeat_front_addr":"172.21.15.5:6809/3901761126","state":["exists","up"]},{"osd":2,"uuid":"c3e8bb76-e382-439d-b380-62ef92f1255a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6817","nonce":502968448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6819","nonce":502968448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6823","nonce":502968448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6821","nonce":502968448}]},"public_addr":"172.21.15.5:6817/502968448","cluster_addr":"172.21.15.5:6819/502968448","heartbeat_back_addr":"172.21.15.5:6823/502968448","heartbeat_front_addr":"172.21.15.5:6821/502968448","state":["exists","up"]},{"osd":3,"uuid":"e8c94625-f2cd-4eba-b53d-2e7356e1c02a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6825","nonce":2425932735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6827","nonce":2425932735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6831","nonce":2425932735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6829","nonce":2425932735}]},"public_addr":"172.21.15.5:6825/2425932735","cluster_addr":"172.21.15.5:6827/2425932735","heartbeat_back_addr":"172.21.15.5:6831/2425932735","heartbeat_front_addr":"172.21.15.5:6829/2425932735","state":["exists","up"]},{"osd":4,"uuid":"0099dd52-6265-4d32-82e4-3ef16e73add0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6817","nonce":976514285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6819","nonce":976514285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6823","nonce":976514285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6821","nonce":976514285}]},"public_addr":"172.21.15.80:6817/976514285","cluster_addr":"172.21.15.80:6819/976514285","heartbeat_back_addr":"172.21.15.80:6823/976514285","heartbeat_front_addr":"172.21.15.80:6821/976514285","state":["exists","up"]},{"osd":5,"uuid":"ed10c84f-8c39-41cc-9b64-7c482a94a48e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6825","nonce":627153447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6827","nonce":627153447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6831","nonce":627153447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6829","nonce":627153447}]},"public_addr":"172.21.15.80:6825/627153447","cluster_addr":"172.21.15.80:6827/627153447","heartbeat_back_addr":"172.21.15.80:6831/627153447","heartbeat_front_addr":"172.21.15.80:6829/627153447","state":["exists","up"]},{"osd":6,"uuid":"c7a3538a-7048-4fba-8781-b440f03c25db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6803","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6805","nonce":488342527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6807","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6809","nonce":488342527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6815","nonce":488342527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6811","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6813","nonce":488342527}]},"public_addr":"172.21.15.80:6805/488342527","cluster_addr":"172.21.15.80:6809/488342527","heartbeat_back_addr":"172.21.15.80:6815/488342527","heartbeat_front_addr":"172.21.15.80:6813/488342527","state":["exists","up"]},{"osd":7,"uuid":"f9418104-3a97-4110-bc3a-1f8151d0f757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6801","nonce":982831666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6804","nonce":982831666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6812","nonce":982831666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6808","nonce":982831666}]},"public_addr":"172.21.15.80:6801/982831666","cluster_addr":"172.21.15.80:6804/982831666","heartbeat_back_addr":"172.21.15.80:6812/982831666","heartbeat_front_addr":"172.21.15.80:6808/982831666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.970828+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.858116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.987556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.864603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.756293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.591502+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:34:09.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.692+0000 7f5d00628640 1 -- 172.21.15.5:0/1405479296 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5cdc067690 msgr2=0x7f5cdc069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:09.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.692+0000 7f5d00628640 1 --2- 172.21.15.5:0/1405479296 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5cdc067690 0x7f5cdc069b50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4001ec0 tx=0x7f5cf4009040 comp rx=0 tx=0).stop 2024-03-18T15:34:09.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.692+0000 7f5d00628640 1 -- 172.21.15.5:0/1405479296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cf8149110 msgr2=0x7f5cf8146cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:09.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.692+0000 7f5d00628640 1 --2- 172.21.15.5:0/1405479296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cf8149110 0x7f5cf8146cf0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f5ce800f910 tx=0x7f5ce800fdd0 comp rx=0 tx=0).stop 2024-03-18T15:34:09.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.692+0000 7f5d00628640 1 -- 172.21.15.5:0/1405479296 shutdown_connections 2024-03-18T15:34:09.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.692+0000 7f5d00628640 1 --2- 172.21.15.5:0/1405479296 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5cdc067690 0x7f5cdc069b50 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:09.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.692+0000 7f5d00628640 1 --2- 172.21.15.5:0/1405479296 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5cf8132c30 0x7f5cf81330e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:09.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.692+0000 7f5d00628640 1 --2- 172.21.15.5:0/1405479296 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5cf81481a0 0x7f5cf8147230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:09.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.692+0000 7f5d00628640 1 --2- 172.21.15.5:0/1405479296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cf8149110 0x7f5cf8146cf0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:09.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.692+0000 7f5d00628640 1 -- 172.21.15.5:0/1405479296 >> 172.21.15.5:0/1405479296 conn(0x7f5cf80b5670 msgr2=0x7f5cf80b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:09.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.692+0000 7f5d00628640 1 -- 172.21.15.5:0/1405479296 shutdown_connections 2024-03-18T15:34:09.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.692+0000 7f5d00628640 1 -- 172.21.15.5:0/1405479296 wait complete. 2024-03-18T15:34:09.711 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T15:33:28.963186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-18T15:33:40.533646+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-18T15:33:41.323519+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T15:34:09.712 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T15:34:09.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.840+0000 7ff0bf4a3640 1 Processor -- start 2024-03-18T15:34:09.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.840+0000 7ff0bf4a3640 1 -- start start 2024-03-18T15:34:09.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.840+0000 7ff0bf4a3640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff0b80b6d40 0x7ff0b8148a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:09.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.840+0000 7ff0bf4a3640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff0b80b4e40 0x7ff0b814fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:09.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.840+0000 7ff0bf4a3640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0b8149420 con 0x7ff0b8148f90 2024-03-18T15:34:09.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.840+0000 7ff0bf4a3640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7ff0b80b78e0 con 0x7ff0b80b6d40 2024-03-18T15:34:09.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.840+0000 7ff0bf4a3640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7ff0b80b7120 con 0x7ff0b80b4e40 2024-03-18T15:34:09.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.840+0000 7ff0bd23f640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff0b80b6d40 0x7ff0b8148a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:09.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.840+0000 7ff0bca3e640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff0b8148f90 0x7ff0b80b4730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60180/0 (socket says 172.21.15.5:60180) 2024-03-18T15:34:09.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.840+0000 7ff0bca3e640 1 -- 172.21.15.5:0/2649078153 learned_addr learned my addr 172.21.15.5:0/2649078153 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:09.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.840+0000 7ff0bda40640 1 --2- 172.21.15.5:0/2649078153 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff0b80b4e40 0x7ff0b814fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:09.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.840+0000 7ff0bd23f640 1 -- 172.21.15.5:0/2649078153 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff0b80b4e40 msgr2=0x7ff0b814fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:09.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.840+0000 7ff0bd23f640 1 --2- 172.21.15.5:0/2649078153 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff0b80b4e40 0x7ff0b814fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:09.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.840+0000 7ff0bd23f640 1 -- 172.21.15.5:0/2649078153 >> v1:172.21.15.5:6789/0 conn(0x7ff0b8148f90 legacy=0x7ff0b80b4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:09.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.840+0000 7ff0bd23f640 1 -- 172.21.15.5:0/2649078153 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0b80b5e20 con 0x7ff0b80b6d40 2024-03-18T15:34:09.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.844+0000 7ff0bd23f640 1 --2- 172.21.15.5:0/2649078153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0b80b6d40 0x7ff0b8148a50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff0b40097c0 tx=0x7ff0b401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aab124f0fd1ebcf1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:09.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.844+0000 7ff0affff640 1 -- 172.21.15.5:0/2649078153 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b40b0070 con 0x7ff0b80b6d40 2024-03-18T15:34:09.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.844+0000 7ff0affff640 1 -- 172.21.15.5:0/2649078153 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0b40ab600 con 0x7ff0b80b6d40 2024-03-18T15:34:09.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.844+0000 7ff0affff640 1 -- 172.21.15.5:0/2649078153 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b40ab9d0 con 0x7ff0b80b6d40 2024-03-18T15:34:09.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.844+0000 7ff0bf4a3640 1 -- 172.21.15.5:0/2649078153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0b80b6d40 msgr2=0x7ff0b8148a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:09.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.844+0000 7ff0bf4a3640 1 --2- 172.21.15.5:0/2649078153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0b80b6d40 0x7ff0b8148a50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff0b40097c0 tx=0x7ff0b401c410 comp rx=0 tx=0).stop 2024-03-18T15:34:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.844+0000 7ff0bf4a3640 1 -- 172.21.15.5:0/2649078153 shutdown_connections 2024-03-18T15:34:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.844+0000 7ff0bf4a3640 1 --2- 172.21.15.5:0/2649078153 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff0b80b4e40 0x7ff0b814fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.844+0000 7ff0bf4a3640 1 --2- 172.21.15.5:0/2649078153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0b80b6d40 0x7ff0b8148a50 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.844+0000 7ff0bf4a3640 1 -- 172.21.15.5:0/2649078153 >> 172.21.15.5:0/2649078153 conn(0x7ff0b8066fc0 msgr2=0x7ff0b80673d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.844+0000 7ff0bf4a3640 1 -- 172.21.15.5:0/2649078153 shutdown_connections 2024-03-18T15:34:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.844+0000 7ff0bf4a3640 1 -- 172.21.15.5:0/2649078153 wait complete. 2024-03-18T15:34:09.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.844+0000 7ff0bf4a3640 1 Processor -- start 2024-03-18T15:34:09.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.844+0000 7ff0bf4a3640 1 -- start start 2024-03-18T15:34:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.844+0000 7ff0bf4a3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0b80b4e40 0x7ff0b813ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.844+0000 7ff0bf4a3640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff0b80b6d40 0x7ff0b8141f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.844+0000 7ff0bf4a3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff0b8148f90 0x7ff0b8141190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.844+0000 7ff0bf4a3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b8152d70 con 0x7ff0b80b4e40 2024-03-18T15:34:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.844+0000 7ff0bf4a3640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff0b8152bf0 con 0x7ff0b8148f90 2024-03-18T15:34:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.844+0000 7ff0bf4a3640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7ff0b8152ef0 con 0x7ff0b80b6d40 2024-03-18T15:34:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.844+0000 7ff0bd23f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0b80b4e40 0x7ff0b813ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.844+0000 7ff0bd23f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0b80b4e40 0x7ff0b813ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50278/0 (socket says 172.21.15.5:50278) 2024-03-18T15:34:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.844+0000 7ff0bd23f640 1 -- 172.21.15.5:0/1586805769 learned_addr learned my addr 172.21.15.5:0/1586805769 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.844+0000 7ff0bca3e640 1 --2- 172.21.15.5:0/1586805769 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff0b80b6d40 0x7ff0b8141f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.844+0000 7ff0bda40640 1 --2- 172.21.15.5:0/1586805769 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff0b8148f90 0x7ff0b8141190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.844+0000 7ff0bd23f640 1 -- 172.21.15.5:0/1586805769 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff0b80b6d40 msgr2=0x7ff0b8141f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.844+0000 7ff0bd23f640 1 --2- 172.21.15.5:0/1586805769 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff0b80b6d40 0x7ff0b8141f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.844+0000 7ff0bd23f640 1 -- 172.21.15.5:0/1586805769 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff0b8148f90 msgr2=0x7ff0b8141190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:09.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.844+0000 7ff0bd23f640 1 --2- 172.21.15.5:0/1586805769 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff0b8148f90 0x7ff0b8141190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:09.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.844+0000 7ff0bd23f640 1 -- 172.21.15.5:0/1586805769 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0b81428a0 con 0x7ff0b80b4e40 2024-03-18T15:34:09.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.844+0000 7ff0bca3e640 1 --2- 172.21.15.5:0/1586805769 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff0b80b6d40 0x7ff0b8141f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:34:09.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.844+0000 7ff0bd23f640 1 --2- 172.21.15.5:0/1586805769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0b80b4e40 0x7ff0b813ffb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff0b401a040 tx=0x7ff0b40191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:09.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.844+0000 7ff0adffb640 1 -- 172.21.15.5:0/1586805769 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b40b8070 con 0x7ff0b80b4e40 2024-03-18T15:34:09.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.844+0000 7ff0adffb640 1 -- 172.21.15.5:0/1586805769 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0b40b4410 con 0x7ff0b80b4e40 2024-03-18T15:34:09.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.844+0000 7ff0bf4a3640 1 -- 172.21.15.5:0/1586805769 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0b8142b80 con 0x7ff0b80b4e40 2024-03-18T15:34:09.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.844+0000 7ff0adffb640 1 -- 172.21.15.5:0/1586805769 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0b40b0040 con 0x7ff0b80b4e40 2024-03-18T15:34:09.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.844+0000 7ff0bf4a3640 1 -- 172.21.15.5:0/1586805769 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0b81416d0 con 0x7ff0b80b4e40 2024-03-18T15:34:09.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.848+0000 7ff0bf4a3640 1 -- 172.21.15.5:0/1586805769 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0b81404f0 con 0x7ff0b80b4e40 2024-03-18T15:34:09.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.848+0000 7ff0adffb640 1 -- 172.21.15.5:0/1586805769 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7ff0b40bb070 con 0x7ff0b80b4e40 2024-03-18T15:34:09.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.848+0000 7ff0adffb640 1 --2- 172.21.15.5:0/1586805769 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7ff09c067660 0x7ff09c069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:09.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.848+0000 7ff0adffb640 1 -- 172.21.15.5:0/1586805769 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff0b411f410 con 0x7ff0b80b4e40 2024-03-18T15:34:09.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.848+0000 7ff0bca3e640 1 --2- 172.21.15.5:0/1586805769 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7ff09c067660 0x7ff09c069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:09.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.848+0000 7ff0bca3e640 1 --2- 172.21.15.5:0/1586805769 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7ff09c067660 0x7ff09c069b20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff0b80bbcc0 tx=0x7ff0a8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:09.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:09.852+0000 7ff0adffb640 1 -- 172.21.15.5:0/1586805769 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7ff0b4016090 con 0x7ff0b80b4e40 2024-03-18T15:34:10.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.008+0000 7ff0bf4a3640 1 -- 172.21.15.5:0/1586805769 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff0b804f550 con 0x7ff0b80b4e40 2024-03-18T15:34:10.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.008+0000 7ff0adffb640 1 -- 172.21.15.5:0/1586805769 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7ff0b40f0120 con 0x7ff0b80b4e40 2024-03-18T15:34:10.012 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-03-18T15:34:10.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.012+0000 7ff0bf4a3640 1 -- 172.21.15.5:0/1586805769 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7ff09c067660 msgr2=0x7ff09c069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:10.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.012+0000 7ff0bf4a3640 1 --2- 172.21.15.5:0/1586805769 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7ff09c067660 0x7ff09c069b20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff0b80bbcc0 tx=0x7ff0a8009040 comp rx=0 tx=0).stop 2024-03-18T15:34:10.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.012+0000 7ff0bf4a3640 1 -- 172.21.15.5:0/1586805769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0b80b4e40 msgr2=0x7ff0b813ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:10.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.012+0000 7ff0bf4a3640 1 --2- 172.21.15.5:0/1586805769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0b80b4e40 0x7ff0b813ffb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff0b401a040 tx=0x7ff0b40191b0 comp rx=0 tx=0).stop 2024-03-18T15:34:10.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.012+0000 7ff0bf4a3640 1 -- 172.21.15.5:0/1586805769 shutdown_connections 2024-03-18T15:34:10.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.012+0000 7ff0bf4a3640 1 --2- 172.21.15.5:0/1586805769 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7ff09c067660 0x7ff09c069b20 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:10.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.012+0000 7ff0bf4a3640 1 --2- 172.21.15.5:0/1586805769 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff0b8148f90 0x7ff0b8141190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:10.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.012+0000 7ff0bf4a3640 1 --2- 172.21.15.5:0/1586805769 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff0b80b6d40 0x7ff0b8141f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:10.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.012+0000 7ff0bf4a3640 1 --2- 172.21.15.5:0/1586805769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0b80b4e40 0x7ff0b813ffb0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:10.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.012+0000 7ff0bf4a3640 1 -- 172.21.15.5:0/1586805769 >> 172.21.15.5:0/1586805769 conn(0x7ff0b8066fc0 msgr2=0x7ff0b80b5410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:10.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.012+0000 7ff0bf4a3640 1 -- 172.21.15.5:0/1586805769 shutdown_connections 2024-03-18T15:34:10.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.012+0000 7ff0bf4a3640 1 -- 172.21.15.5:0/1586805769 wait complete. 2024-03-18T15:34:10.029 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-18T15:34:10.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.152+0000 7fe18ef0c640 1 Processor -- start 2024-03-18T15:34:10.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.152+0000 7fe18ef0c640 1 -- start start 2024-03-18T15:34:10.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.152+0000 7fe18ef0c640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe18813f6a0 0x7fe1881513a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:10.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.152+0000 7fe18ef0c640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1880b2ab0 con 0x7fe18813f260 2024-03-18T15:34:10.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.152+0000 7fe18ef0c640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe18813fb10 con 0x7fe18806d2d0 2024-03-18T15:34:10.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.152+0000 7fe18ef0c640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fe1880728c0 con 0x7fe18813f6a0 2024-03-18T15:34:10.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.152+0000 7fe187fff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fe18813f260 0x7fe18814dc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:60196/0 (socket says 172.21.15.5:60196) 2024-03-18T15:34:10.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.152+0000 7fe187fff640 1 -- 172.21.15.5:0/820521341 learned_addr learned my addr 172.21.15.5:0/820521341 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:10.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.152+0000 7fe18d4a9640 1 --2- 172.21.15.5:0/820521341 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe18813f6a0 0x7fe1881513a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:10.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.152+0000 7fe1877fe640 1 -- 172.21.15.5:0/820521341 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1343696934 0 0) 0x7fe1880b2ab0 con 0x7fe18813f260 2024-03-18T15:34:10.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.152+0000 7fe1877fe640 1 -- 172.21.15.5:0/820521341 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe170003530 con 0x7fe18813f260 2024-03-18T15:34:10.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.152+0000 7fe1877fe640 1 -- 172.21.15.5:0/820521341 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3390138683 0 0) 0x7fe170003530 con 0x7fe18813f260 2024-03-18T15:34:10.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.152+0000 7fe1877fe640 1 -- 172.21.15.5:0/820521341 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe18813f6a0 msgr2=0x7fe1881513a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:10.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.152+0000 7fe1877fe640 1 --2- 172.21.15.5:0/820521341 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe18813f6a0 0x7fe1881513a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:10.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.152+0000 7fe1877fe640 1 -- 172.21.15.5:0/820521341 >> v1:172.21.15.80:6789/0 conn(0x7fe18806d2d0 legacy=0x7fe1880b23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:10.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.152+0000 7fe1877fe640 1 -- 172.21.15.5:0/820521341 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1880b0ea0 con 0x7fe18813f260 2024-03-18T15:34:10.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.152+0000 7fe1877fe640 1 -- 172.21.15.5:0/820521341 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3837549003 0 0) 0x7fe17c001680 con 0x7fe18813f260 2024-03-18T15:34:10.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.152+0000 7fe1877fe640 1 -- 172.21.15.5:0/820521341 >> v1:172.21.15.5:6789/0 conn(0x7fe18813f260 legacy=0x7fe18814dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:10.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.152+0000 7fe1877fe640 1 --2- 172.21.15.5:0/820521341 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe18813f6a0 0x7fe170003af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:10.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.152+0000 7fe1877fe640 1 --2- 172.21.15.5:0/820521341 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe170004070 0x7fe170004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:10.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.152+0000 7fe1877fe640 1 --2- 172.21.15.5:0/820521341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe170004d50 0x7fe1700051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:10.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.152+0000 7fe1877fe640 1 -- 172.21.15.5:0/820521341 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe1880728c0 con 0x7fe170004d50 2024-03-18T15:34:10.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.152+0000 7fe1877fe640 1 -- 172.21.15.5:0/820521341 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe17000bcb0 con 0x7fe170004070 2024-03-18T15:34:10.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.152+0000 7fe1877fe640 1 -- 172.21.15.5:0/820521341 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fe17000beb0 con 0x7fe18813f6a0 2024-03-18T15:34:10.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.152+0000 7fe18d4a9640 1 --2- 172.21.15.5:0/820521341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe170004d50 0x7fe1700051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:10.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.152+0000 7fe187fff640 1 --2- 172.21.15.5:0/820521341 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe170004070 0x7fe170004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:10.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.152+0000 7fe18cca8640 1 --2- 172.21.15.5:0/820521341 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe18813f6a0 0x7fe170003af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:10.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.152+0000 7fe18d4a9640 1 -- 172.21.15.5:0/820521341 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe18813f6a0 msgr2=0x7fe170003af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:10.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.152+0000 7fe18d4a9640 1 --2- 172.21.15.5:0/820521341 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe18813f6a0 0x7fe170003af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:10.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.152+0000 7fe18d4a9640 1 -- 172.21.15.5:0/820521341 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe170004070 msgr2=0x7fe170004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:10.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.152+0000 7fe18d4a9640 1 --2- 172.21.15.5:0/820521341 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe170004070 0x7fe170004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:10.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.152+0000 7fe18d4a9640 1 -- 172.21.15.5:0/820521341 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe170003530 con 0x7fe170004d50 2024-03-18T15:34:10.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.152+0000 7fe18cca8640 1 --2- 172.21.15.5:0/820521341 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe18813f6a0 0x7fe170003af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:10.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.156+0000 7fe18d4a9640 1 --2- 172.21.15.5:0/820521341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe170004d50 0x7fe1700051d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe180009fc0 tx=0x7fe18001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d65a5e5634e241 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:10.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.156+0000 7fe1877fe640 1 -- 172.21.15.5:0/820521341 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1800b2070 con 0x7fe170004d50 2024-03-18T15:34:10.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.156+0000 7fe1877fe640 1 -- 172.21.15.5:0/820521341 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1800a5b80 con 0x7fe170004d50 2024-03-18T15:34:10.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.156+0000 7fe1877fe640 1 -- 172.21.15.5:0/820521341 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe180009c10 con 0x7fe170004d50 2024-03-18T15:34:10.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.156+0000 7fe18ef0c640 1 -- 172.21.15.5:0/820521341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe170004d50 msgr2=0x7fe1700051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:10.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.156+0000 7fe18ef0c640 1 --2- 172.21.15.5:0/820521341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe170004d50 0x7fe1700051d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe180009fc0 tx=0x7fe18001cc80 comp rx=0 tx=0).stop 2024-03-18T15:34:10.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.156+0000 7fe18d4a9640 1 -- 172.21.15.5:0/820521341 reap_dead start 2024-03-18T15:34:10.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.156+0000 7fe18ef0c640 1 -- 172.21.15.5:0/820521341 shutdown_connections 2024-03-18T15:34:10.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.156+0000 7fe18ef0c640 1 -- 172.21.15.5:0/820521341 >> 172.21.15.5:0/820521341 conn(0x7fe1880b5670 msgr2=0x7fe1880b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:10.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.156+0000 7fe18ef0c640 1 -- 172.21.15.5:0/820521341 shutdown_connections 2024-03-18T15:34:10.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.156+0000 7fe18ef0c640 1 -- 172.21.15.5:0/820521341 wait complete. 2024-03-18T15:34:10.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.156+0000 7fe18ef0c640 1 Processor -- start 2024-03-18T15:34:10.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.156+0000 7fe18ef0c640 1 -- start start 2024-03-18T15:34:10.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.156+0000 7fe18ef0c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe188144e20 0x7fe1881db180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:10.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.156+0000 7fe18ef0c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe188145f10 0x7fe1881dde80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:10.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.156+0000 7fe18ef0c640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1881de3c0 0x7fe1881de840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:10.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.156+0000 7fe18ef0c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe1880b1c30 con 0x7fe188144e20 2024-03-18T15:34:10.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.156+0000 7fe18ef0c640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe1880b0bc0 con 0x7fe188145f10 2024-03-18T15:34:10.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.156+0000 7fe18ef0c640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fe1880b0070 con 0x7fe1881de3c0 2024-03-18T15:34:10.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.156+0000 7fe18cca8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe188144e20 0x7fe1881db180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:10.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.156+0000 7fe18cca8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe188144e20 0x7fe1881db180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50288/0 (socket says 172.21.15.5:50288) 2024-03-18T15:34:10.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.156+0000 7fe18cca8640 1 -- 172.21.15.5:0/3555655717 learned_addr learned my addr 172.21.15.5:0/3555655717 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:10.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.156+0000 7fe18d4a9640 1 --2- 172.21.15.5:0/3555655717 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1881de3c0 0x7fe1881de840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:10.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.156+0000 7fe187fff640 1 --2- 172.21.15.5:0/3555655717 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe188145f10 0x7fe1881dde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:10.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.156+0000 7fe18cca8640 1 -- 172.21.15.5:0/3555655717 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1881de3c0 msgr2=0x7fe1881de840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:10.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.156+0000 7fe18cca8640 1 --2- 172.21.15.5:0/3555655717 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1881de3c0 0x7fe1881de840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:10.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.156+0000 7fe18cca8640 1 -- 172.21.15.5:0/3555655717 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe188145f10 msgr2=0x7fe1881dde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:10.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.156+0000 7fe18cca8640 1 --2- 172.21.15.5:0/3555655717 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe188145f10 0x7fe1881dde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:10.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.156+0000 7fe18cca8640 1 -- 172.21.15.5:0/3555655717 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe188145200 con 0x7fe188144e20 2024-03-18T15:34:10.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.160+0000 7fe18cca8640 1 --2- 172.21.15.5:0/3555655717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe188144e20 0x7fe1881db180 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe17800d7a0 tx=0x7fe17800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:10.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.160+0000 7fe1857fa640 1 -- 172.21.15.5:0/3555655717 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe178005270 con 0x7fe188144e20 2024-03-18T15:34:10.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.160+0000 7fe1857fa640 1 -- 172.21.15.5:0/3555655717 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe178017070 con 0x7fe188144e20 2024-03-18T15:34:10.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.160+0000 7fe18ef0c640 1 -- 172.21.15.5:0/3555655717 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1880b8490 con 0x7fe188144e20 2024-03-18T15:34:10.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.160+0000 7fe1857fa640 1 -- 172.21.15.5:0/3555655717 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17800b6b0 con 0x7fe188144e20 2024-03-18T15:34:10.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.160+0000 7fe18ef0c640 1 -- 172.21.15.5:0/3555655717 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe18806f340 con 0x7fe188144e20 2024-03-18T15:34:10.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.160+0000 7fe18ef0c640 1 -- 172.21.15.5:0/3555655717 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe18804f550 con 0x7fe188144e20 2024-03-18T15:34:10.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.160+0000 7fe1857fa640 1 -- 172.21.15.5:0/3555655717 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fe17800b850 con 0x7fe188144e20 2024-03-18T15:34:10.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.160+0000 7fe1857fa640 1 --2- 172.21.15.5:0/3555655717 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe174067690 0x7fe174069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:10.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.160+0000 7fe1857fa640 1 -- 172.21.15.5:0/3555655717 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe178085720 con 0x7fe188144e20 2024-03-18T15:34:10.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.160+0000 7fe187fff640 1 --2- 172.21.15.5:0/3555655717 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe174067690 0x7fe174069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:10.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.160+0000 7fe187fff640 1 --2- 172.21.15.5:0/3555655717 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe174067690 0x7fe174069b50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe17c001060 tx=0x7fe17c000ff0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:10.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.164+0000 7fe1857fa640 1 -- 172.21.15.5:0/3555655717 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fe178056430 con 0x7fe188144e20 2024-03-18T15:34:10.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.324+0000 7fe18ef0c640 1 -- 172.21.15.5:0/3555655717 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe18813d3b0 con 0x7fe188144e20 2024-03-18T15:34:10.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.324+0000 7fe1857fa640 1 -- 172.21.15.5:0/3555655717 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7fe178055d30 con 0x7fe188144e20 2024-03-18T15:34:10.327 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-03-18T15:34:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.324+0000 7fe18ef0c640 1 -- 172.21.15.5:0/3555655717 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe174067690 msgr2=0x7fe174069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.324+0000 7fe18ef0c640 1 --2- 172.21.15.5:0/3555655717 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe174067690 0x7fe174069b50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe17c001060 tx=0x7fe17c000ff0 comp rx=0 tx=0).stop 2024-03-18T15:34:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.324+0000 7fe18ef0c640 1 -- 172.21.15.5:0/3555655717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe188144e20 msgr2=0x7fe1881db180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.324+0000 7fe18ef0c640 1 --2- 172.21.15.5:0/3555655717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe188144e20 0x7fe1881db180 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe17800d7a0 tx=0x7fe17800dc60 comp rx=0 tx=0).stop 2024-03-18T15:34:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.328+0000 7fe18ef0c640 1 -- 172.21.15.5:0/3555655717 shutdown_connections 2024-03-18T15:34:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.328+0000 7fe18ef0c640 1 --2- 172.21.15.5:0/3555655717 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe174067690 0x7fe174069b50 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.328+0000 7fe18ef0c640 1 --2- 172.21.15.5:0/3555655717 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe1881de3c0 0x7fe1881de840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:10.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.328+0000 7fe18ef0c640 1 --2- 172.21.15.5:0/3555655717 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe188145f10 0x7fe1881dde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:10.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.328+0000 7fe18ef0c640 1 --2- 172.21.15.5:0/3555655717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe188144e20 0x7fe1881db180 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:10.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.328+0000 7fe18ef0c640 1 -- 172.21.15.5:0/3555655717 >> 172.21.15.5:0/3555655717 conn(0x7fe1880b5670 msgr2=0x7fe18806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:10.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.328+0000 7fe18ef0c640 1 -- 172.21.15.5:0/3555655717 shutdown_connections 2024-03-18T15:34:10.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.328+0000 7fe18ef0c640 1 -- 172.21.15.5:0/3555655717 wait complete. 2024-03-18T15:34:10.346 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-18T15:34:10.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.468+0000 7fd420617640 1 Processor -- start 2024-03-18T15:34:10.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.468+0000 7fd420617640 1 -- start start 2024-03-18T15:34:10.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.468+0000 7fd420617640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd4180aec00 0x7fd4180aefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:10.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.468+0000 7fd420617640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd418143bd0 0x7fd418143ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:10.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.468+0000 7fd420617640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd418147060 0x7fd4181474b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:10.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.468+0000 7fd420617640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fd4180ae090 con 0x7fd418143bd0 2024-03-18T15:34:10.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.468+0000 7fd420617640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fd4180b0350 con 0x7fd4180aec00 2024-03-18T15:34:10.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.468+0000 7fd420617640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fd4180add60 con 0x7fd418147060 2024-03-18T15:34:10.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.468+0000 7fd41dbb2640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd418143bd0 0x7fd418143ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:10.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.468+0000 7fd41dbb2640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd418143bd0 0x7fd418143ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50300/0 (socket says 172.21.15.5:50300) 2024-03-18T15:34:10.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.468+0000 7fd41dbb2640 1 -- 172.21.15.5:0/3178316129 learned_addr learned my addr 172.21.15.5:0/3178316129 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:10.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.468+0000 7fd41e3b3640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd4180aec00 0x7fd4180aefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:10.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.468+0000 7fd41ebb4640 1 --2- 172.21.15.5:0/3178316129 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd418147060 0x7fd4181474b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:10.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.468+0000 7fd41dbb2640 1 -- 172.21.15.5:0/3178316129 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd418147060 msgr2=0x7fd4181474b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:10.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.468+0000 7fd41dbb2640 1 --2- 172.21.15.5:0/3178316129 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd418147060 0x7fd4181474b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:10.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.468+0000 7fd41dbb2640 1 -- 172.21.15.5:0/3178316129 >> v2:172.21.15.80:3300/0 conn(0x7fd4180aec00 msgr2=0x7fd4180aefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:10.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.468+0000 7fd41dbb2640 1 --2- 172.21.15.5:0/3178316129 >> v2:172.21.15.80:3300/0 conn(0x7fd4180aec00 0x7fd4180aefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:10.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.468+0000 7fd41dbb2640 1 -- 172.21.15.5:0/3178316129 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4180b0630 con 0x7fd418143bd0 2024-03-18T15:34:10.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.472+0000 7fd41dbb2640 1 --2- 172.21.15.5:0/3178316129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd418143bd0 0x7fd418143ff0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd41806ef00 tx=0x7fd40801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=461a8b48d8f3aaf6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:10.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.472+0000 7fd41d3b1640 1 -- 172.21.15.5:0/3178316129 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4080b0070 con 0x7fd418143bd0 2024-03-18T15:34:10.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.472+0000 7fd41d3b1640 1 -- 172.21.15.5:0/3178316129 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4080ab600 con 0x7fd418143bd0 2024-03-18T15:34:10.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.472+0000 7fd41d3b1640 1 -- 172.21.15.5:0/3178316129 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4080ab9d0 con 0x7fd418143bd0 2024-03-18T15:34:10.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.472+0000 7fd420617640 1 -- 172.21.15.5:0/3178316129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd418143bd0 msgr2=0x7fd418143ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:10.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.472+0000 7fd420617640 1 --2- 172.21.15.5:0/3178316129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd418143bd0 0x7fd418143ff0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd41806ef00 tx=0x7fd40801c520 comp rx=0 tx=0).stop 2024-03-18T15:34:10.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.472+0000 7fd420617640 1 -- 172.21.15.5:0/3178316129 shutdown_connections 2024-03-18T15:34:10.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.472+0000 7fd420617640 1 --2- 172.21.15.5:0/3178316129 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd418147060 0x7fd4181474b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:10.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.472+0000 7fd420617640 1 --2- 172.21.15.5:0/3178316129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd418143bd0 0x7fd418143ff0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:10.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.472+0000 7fd420617640 1 --2- 172.21.15.5:0/3178316129 >> v2:172.21.15.80:3300/0 conn(0x7fd4180aec00 0x7fd4180aefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:10.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.472+0000 7fd420617640 1 -- 172.21.15.5:0/3178316129 >> 172.21.15.5:0/3178316129 conn(0x7fd4180b5670 msgr2=0x7fd4180b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:10.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.472+0000 7fd420617640 1 -- 172.21.15.5:0/3178316129 shutdown_connections 2024-03-18T15:34:10.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.472+0000 7fd420617640 1 -- 172.21.15.5:0/3178316129 wait complete. 2024-03-18T15:34:10.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.472+0000 7fd420617640 1 Processor -- start 2024-03-18T15:34:10.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.472+0000 7fd420617640 1 -- start start 2024-03-18T15:34:10.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.472+0000 7fd420617640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd4180aec00 0x7fd41813b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:10.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.472+0000 7fd420617640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd418143bd0 0x7fd4181dd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:10.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.472+0000 7fd420617640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd418147060 0x7fd4181ddd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:10.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.472+0000 7fd420617640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd418140950 con 0x7fd418147060 2024-03-18T15:34:10.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.472+0000 7fd420617640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd4180ae4f0 con 0x7fd418143bd0 2024-03-18T15:34:10.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.472+0000 7fd420617640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fd418140ad0 con 0x7fd4180aec00 2024-03-18T15:34:10.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.472+0000 7fd41ebb4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd418147060 0x7fd4181ddd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:10.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.472+0000 7fd41ebb4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd418147060 0x7fd4181ddd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50314/0 (socket says 172.21.15.5:50314) 2024-03-18T15:34:10.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.472+0000 7fd41ebb4640 1 -- 172.21.15.5:0/1297799930 learned_addr learned my addr 172.21.15.5:0/1297799930 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:10.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.472+0000 7fd41dbb2640 1 --2- 172.21.15.5:0/1297799930 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd418143bd0 0x7fd4181dd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:10.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.472+0000 7fd41e3b3640 1 --2- 172.21.15.5:0/1297799930 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd4180aec00 0x7fd41813b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:10.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.476+0000 7fd41ebb4640 1 -- 172.21.15.5:0/1297799930 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd4180aec00 msgr2=0x7fd41813b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:10.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.476+0000 7fd41ebb4640 1 --2- 172.21.15.5:0/1297799930 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd4180aec00 0x7fd41813b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:10.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.476+0000 7fd41ebb4640 1 -- 172.21.15.5:0/1297799930 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd418143bd0 msgr2=0x7fd4181dd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:10.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.476+0000 7fd41ebb4640 1 --2- 172.21.15.5:0/1297799930 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd418143bd0 0x7fd4181dd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:10.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.476+0000 7fd41ebb4640 1 -- 172.21.15.5:0/1297799930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4180b2340 con 0x7fd418147060 2024-03-18T15:34:10.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.476+0000 7fd41dbb2640 1 --2- 172.21.15.5:0/1297799930 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd418143bd0 0x7fd4181dd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:34:10.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.476+0000 7fd41ebb4640 1 --2- 172.21.15.5:0/1297799930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd418147060 0x7fd4181ddd00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd410009d30 tx=0x7fd41000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:10.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.476+0000 7fd40effd640 1 -- 172.21.15.5:0/1297799930 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd410018020 con 0x7fd418147060 2024-03-18T15:34:10.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.476+0000 7fd40effd640 1 -- 172.21.15.5:0/1297799930 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd410014480 con 0x7fd418147060 2024-03-18T15:34:10.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.476+0000 7fd420617640 1 -- 172.21.15.5:0/1297799930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4180b11d0 con 0x7fd418147060 2024-03-18T15:34:10.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.476+0000 7fd40effd640 1 -- 172.21.15.5:0/1297799930 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd410014870 con 0x7fd418147060 2024-03-18T15:34:10.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.476+0000 7fd420617640 1 -- 172.21.15.5:0/1297799930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd41813e730 con 0x7fd418147060 2024-03-18T15:34:10.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.476+0000 7fd420617640 1 -- 172.21.15.5:0/1297799930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4180b0ff0 con 0x7fd418147060 2024-03-18T15:34:10.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.476+0000 7fd40effd640 1 -- 172.21.15.5:0/1297799930 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fd41001b020 con 0x7fd418147060 2024-03-18T15:34:10.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.476+0000 7fd40effd640 1 --2- 172.21.15.5:0/1297799930 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd3fc067690 0x7fd3fc069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:10.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.476+0000 7fd40effd640 1 -- 172.21.15.5:0/1297799930 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fd410086210 con 0x7fd418147060 2024-03-18T15:34:10.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.476+0000 7fd41e3b3640 1 --2- 172.21.15.5:0/1297799930 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd3fc067690 0x7fd3fc069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:10.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.476+0000 7fd41e3b3640 1 --2- 172.21.15.5:0/1297799930 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd3fc067690 0x7fd3fc069b50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd41813c0a0 tx=0x7fd414008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:10.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.480+0000 7fd40effd640 1 -- 172.21.15.5:0/1297799930 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fd4180b0ff0 con 0x7fd418147060 2024-03-18T15:34:10.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.636+0000 7fd420617640 1 -- 172.21.15.5:0/1297799930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd41804f550 con 0x7fd418147060 2024-03-18T15:34:10.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.636+0000 7fd40effd640 1 -- 172.21.15.5:0/1297799930 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7fd410022070 con 0x7fd418147060 2024-03-18T15:34:10.640 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-03-18T15:34:10.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.640+0000 7fd420617640 1 -- 172.21.15.5:0/1297799930 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd3fc067690 msgr2=0x7fd3fc069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:10.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.640+0000 7fd420617640 1 --2- 172.21.15.5:0/1297799930 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd3fc067690 0x7fd3fc069b50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd41813c0a0 tx=0x7fd414008040 comp rx=0 tx=0).stop 2024-03-18T15:34:10.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.640+0000 7fd420617640 1 -- 172.21.15.5:0/1297799930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd418147060 msgr2=0x7fd4181ddd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:10.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.640+0000 7fd420617640 1 --2- 172.21.15.5:0/1297799930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd418147060 0x7fd4181ddd00 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd410009d30 tx=0x7fd41000c490 comp rx=0 tx=0).stop 2024-03-18T15:34:10.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.640+0000 7fd420617640 1 -- 172.21.15.5:0/1297799930 shutdown_connections 2024-03-18T15:34:10.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.640+0000 7fd420617640 1 --2- 172.21.15.5:0/1297799930 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd3fc067690 0x7fd3fc069b50 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:10.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.640+0000 7fd420617640 1 --2- 172.21.15.5:0/1297799930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd418147060 0x7fd4181ddd00 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:10.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.640+0000 7fd420617640 1 --2- 172.21.15.5:0/1297799930 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd418143bd0 0x7fd4181dd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:10.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.640+0000 7fd420617640 1 --2- 172.21.15.5:0/1297799930 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd4180aec00 0x7fd41813b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:10.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.640+0000 7fd420617640 1 -- 172.21.15.5:0/1297799930 >> 172.21.15.5:0/1297799930 conn(0x7fd4180b5670 msgr2=0x7fd418140010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:10.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.640+0000 7fd420617640 1 -- 172.21.15.5:0/1297799930 shutdown_connections 2024-03-18T15:34:10.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.640+0000 7fd420617640 1 -- 172.21.15.5:0/1297799930 wait complete. 2024-03-18T15:34:10.657 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:34:10.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.776+0000 7fc8943c1640 1 Processor -- start 2024-03-18T15:34:10.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.776+0000 7fc8943c1640 1 -- start start 2024-03-18T15:34:10.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.776+0000 7fc8943c1640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc88c06d2d0 0x7fc88c0b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:10.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.776+0000 7fc8943c1640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc88c13f260 0x7fc88c13f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:10.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.776+0000 7fc8943c1640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc88c0b0ea0 con 0x7fc88c1453a0 2024-03-18T15:34:10.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.776+0000 7fc8943c1640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fc88c0728c0 con 0x7fc88c06d2d0 2024-03-18T15:34:10.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.776+0000 7fc8943c1640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fc88c06d6b0 con 0x7fc88c13f260 2024-03-18T15:34:10.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.776+0000 7fc89215d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc88c06d2d0 0x7fc88c0b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:10.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.776+0000 7fc89295e640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fc88c1453a0 0x7fc88c145810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.5:38000/0 (socket says 172.21.15.5:38000) 2024-03-18T15:34:10.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.776+0000 7fc89295e640 1 -- 172.21.15.5:0/1622094041 learned_addr learned my addr 172.21.15.5:0/1622094041 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:10.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.776+0000 7fc89195c640 1 --2- 172.21.15.5:0/1622094041 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc88c13f260 0x7fc88c13f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:10.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.776+0000 7fc89115b640 1 -- 172.21.15.5:0/1622094041 <== mon.1 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 865650984 0 0) 0x7fc88c0b0ea0 con 0x7fc88c1453a0 2024-03-18T15:34:10.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.776+0000 7fc89115b640 1 -- 172.21.15.5:0/1622094041 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc878003530 con 0x7fc88c1453a0 2024-03-18T15:34:10.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.776+0000 7fc89195c640 1 -- 172.21.15.5:0/1622094041 >> v2:172.21.15.80:3300/0 conn(0x7fc88c06d2d0 msgr2=0x7fc88c0b23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:10.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.776+0000 7fc89195c640 1 --2- 172.21.15.5:0/1622094041 >> v2:172.21.15.80:3300/0 conn(0x7fc88c06d2d0 0x7fc88c0b23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:10.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.776+0000 7fc89195c640 1 -- 172.21.15.5:0/1622094041 >> v1:172.21.15.80:6789/0 conn(0x7fc88c1453a0 legacy=0x7fc88c145810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:10.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.776+0000 7fc89195c640 1 -- 172.21.15.5:0/1622094041 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc88c0b0bc0 con 0x7fc88c13f260 2024-03-18T15:34:10.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.780+0000 7fc89195c640 1 --2- 172.21.15.5:0/1622094041 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc88c13f260 0x7fc88c13f6a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc8880097c0 tx=0x7fc88801c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8945e25e7140738 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:10.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.780+0000 7fc89115b640 1 -- 172.21.15.5:0/1622094041 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8880b0070 con 0x7fc88c13f260 2024-03-18T15:34:10.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.780+0000 7fc89115b640 1 -- 172.21.15.5:0/1622094041 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8880ab600 con 0x7fc88c13f260 2024-03-18T15:34:10.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.780+0000 7fc89115b640 1 -- 172.21.15.5:0/1622094041 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8880ab9d0 con 0x7fc88c13f260 2024-03-18T15:34:10.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.780+0000 7fc8943c1640 1 -- 172.21.15.5:0/1622094041 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc88c13f260 msgr2=0x7fc88c13f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:10.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.780+0000 7fc8943c1640 1 --2- 172.21.15.5:0/1622094041 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc88c13f260 0x7fc88c13f6a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc8880097c0 tx=0x7fc88801c410 comp rx=0 tx=0).stop 2024-03-18T15:34:10.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.780+0000 7fc8943c1640 1 -- 172.21.15.5:0/1622094041 shutdown_connections 2024-03-18T15:34:10.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.780+0000 7fc8943c1640 1 --2- 172.21.15.5:0/1622094041 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc88c13f260 0x7fc88c13f6a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:10.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.780+0000 7fc8943c1640 1 --2- 172.21.15.5:0/1622094041 >> v2:172.21.15.80:3300/0 conn(0x7fc88c06d2d0 0x7fc88c0b23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:10.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.780+0000 7fc8943c1640 1 -- 172.21.15.5:0/1622094041 >> 172.21.15.5:0/1622094041 conn(0x7fc88c0b5670 msgr2=0x7fc88c0b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:10.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.780+0000 7fc8943c1640 1 -- 172.21.15.5:0/1622094041 shutdown_connections 2024-03-18T15:34:10.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.780+0000 7fc8943c1640 1 -- 172.21.15.5:0/1622094041 wait complete. 2024-03-18T15:34:10.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.780+0000 7fc8943c1640 1 Processor -- start 2024-03-18T15:34:10.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.780+0000 7fc8943c1640 1 -- start start 2024-03-18T15:34:10.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.780+0000 7fc8943c1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc88c06d2d0 0x7fc88c139890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:10.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.780+0000 7fc8943c1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc88c13f260 0x7fc88c0707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:10.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.780+0000 7fc8943c1640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc88c1453a0 0x7fc88c071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:10.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.780+0000 7fc8943c1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc88c0b52c0 con 0x7fc88c06d2d0 2024-03-18T15:34:10.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.780+0000 7fc8943c1640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc88c0b5140 con 0x7fc88c13f260 2024-03-18T15:34:10.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.780+0000 7fc8943c1640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fc88c072d30 con 0x7fc88c1453a0 2024-03-18T15:34:10.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.780+0000 7fc89215d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc88c06d2d0 0x7fc88c139890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:10.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.780+0000 7fc89215d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc88c06d2d0 0x7fc88c139890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45726/0 (socket says 172.21.15.5:45726) 2024-03-18T15:34:10.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.780+0000 7fc89215d640 1 -- 172.21.15.5:0/2106203607 learned_addr learned my addr 172.21.15.5:0/2106203607 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:10.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.780+0000 7fc89195c640 1 --2- 172.21.15.5:0/2106203607 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc88c13f260 0x7fc88c0707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:10.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.780+0000 7fc89295e640 1 --2- 172.21.15.5:0/2106203607 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc88c1453a0 0x7fc88c071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:10.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.784+0000 7fc89215d640 1 -- 172.21.15.5:0/2106203607 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc88c1453a0 msgr2=0x7fc88c071960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:10.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.784+0000 7fc89215d640 1 --2- 172.21.15.5:0/2106203607 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc88c1453a0 0x7fc88c071960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:10.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.784+0000 7fc89215d640 1 -- 172.21.15.5:0/2106203607 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc88c13f260 msgr2=0x7fc88c0707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:10.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.784+0000 7fc89215d640 1 --2- 172.21.15.5:0/2106203607 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc88c13f260 0x7fc88c0707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:10.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.784+0000 7fc89215d640 1 -- 172.21.15.5:0/2106203607 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc88c070f20 con 0x7fc88c06d2d0 2024-03-18T15:34:10.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.784+0000 7fc89215d640 1 --2- 172.21.15.5:0/2106203607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc88c06d2d0 0x7fc88c139890 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc88000e8a0 tx=0x7fc88000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:10.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.784+0000 7fc87effd640 1 -- 172.21.15.5:0/2106203607 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc880017070 con 0x7fc88c06d2d0 2024-03-18T15:34:10.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.784+0000 7fc87effd640 1 -- 172.21.15.5:0/2106203607 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc88000c940 con 0x7fc88c06d2d0 2024-03-18T15:34:10.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.784+0000 7fc8943c1640 1 -- 172.21.15.5:0/2106203607 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc88c13a2c0 con 0x7fc88c06d2d0 2024-03-18T15:34:10.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.784+0000 7fc87effd640 1 -- 172.21.15.5:0/2106203607 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88000ccf0 con 0x7fc88c06d2d0 2024-03-18T15:34:10.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.784+0000 7fc8943c1640 1 -- 172.21.15.5:0/2106203607 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc88c06fd40 con 0x7fc88c06d2d0 2024-03-18T15:34:10.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.784+0000 7fc8943c1640 1 -- 172.21.15.5:0/2106203607 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc88c04f550 con 0x7fc88c06d2d0 2024-03-18T15:34:10.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.784+0000 7fc87effd640 1 -- 172.21.15.5:0/2106203607 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fc88001a070 con 0x7fc88c06d2d0 2024-03-18T15:34:10.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.784+0000 7fc87effd640 1 --2- 172.21.15.5:0/2106203607 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fc870067660 0x7fc870069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:10.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.784+0000 7fc87effd640 1 -- 172.21.15.5:0/2106203607 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc880085ae0 con 0x7fc88c06d2d0 2024-03-18T15:34:10.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.784+0000 7fc89195c640 1 --2- 172.21.15.5:0/2106203607 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fc870067660 0x7fc870069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:10.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.784+0000 7fc89195c640 1 --2- 172.21.15.5:0/2106203607 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fc870067660 0x7fc870069b20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc8880ad000 tx=0x7fc888008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:10.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.788+0000 7fc87effd640 1 -- 172.21.15.5:0/2106203607 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fc880056080 con 0x7fc88c06d2d0 2024-03-18T15:34:10.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.984+0000 7fc8943c1640 1 -- 172.21.15.5:0/2106203607 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc88c0b88b0 con 0x7fc88c06d2d0 2024-03-18T15:34:10.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.984+0000 7fc87effd640 1 -- 172.21.15.5:0/2106203607 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+3521 (secure 0 0 0) 0x7fc88005a120 con 0x7fc88c06d2d0 2024-03-18T15:34:10.989 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:34:10.989 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/629589896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":629589896},{"type":"v1","addr":"172.21.15.5:6835","nonce":629589896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-18T15:33:42.338926+0000","modified":"2024-03-18T15:33:45.376646+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6833/1236603350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":1236603350},{"type":"v1","addr":"172.21.15.80:6833","nonce":1236603350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4272,"qdb_cluster":[4272]},"id":1}]} 2024-03-18T15:34:10.989 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 8 2024-03-18T15:34:10.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.988+0000 7fc8943c1640 1 -- 172.21.15.5:0/2106203607 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fc870067660 msgr2=0x7fc870069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:10.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.988+0000 7fc8943c1640 1 --2- 172.21.15.5:0/2106203607 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fc870067660 0x7fc870069b20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc8880ad000 tx=0x7fc888008040 comp rx=0 tx=0).stop 2024-03-18T15:34:10.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.988+0000 7fc8943c1640 1 -- 172.21.15.5:0/2106203607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc88c06d2d0 msgr2=0x7fc88c139890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:10.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.988+0000 7fc8943c1640 1 --2- 172.21.15.5:0/2106203607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc88c06d2d0 0x7fc88c139890 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc88000e8a0 tx=0x7fc88000eca0 comp rx=0 tx=0).stop 2024-03-18T15:34:10.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.988+0000 7fc8943c1640 1 -- 172.21.15.5:0/2106203607 shutdown_connections 2024-03-18T15:34:10.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.988+0000 7fc8943c1640 1 --2- 172.21.15.5:0/2106203607 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fc870067660 0x7fc870069b20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:10.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.988+0000 7fc8943c1640 1 --2- 172.21.15.5:0/2106203607 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc88c1453a0 0x7fc88c071960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:10.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.988+0000 7fc8943c1640 1 --2- 172.21.15.5:0/2106203607 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc88c13f260 0x7fc88c0707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:10.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.988+0000 7fc8943c1640 1 --2- 172.21.15.5:0/2106203607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc88c06d2d0 0x7fc88c139890 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:10.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.988+0000 7fc8943c1640 1 -- 172.21.15.5:0/2106203607 >> 172.21.15.5:0/2106203607 conn(0x7fc88c0b5670 msgr2=0x7fc88c13b130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:10.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.988+0000 7fc8943c1640 1 -- 172.21.15.5:0/2106203607 shutdown_connections 2024-03-18T15:34:10.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:10.988+0000 7fc8943c1640 1 -- 172.21.15.5:0/2106203607 wait complete. 2024-03-18T15:34:11.008 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:34:11.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.120+0000 7fb9545b5640 1 Processor -- start 2024-03-18T15:34:11.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.124+0000 7fb9545b5640 1 -- start start 2024-03-18T15:34:11.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.124+0000 7fb9545b5640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb94c0744b0 0x7fb94c0b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:11.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.124+0000 7fb9545b5640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb94c144390 0x7fb94c144800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:11.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.124+0000 7fb9545b5640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb94c141420 0x7fb94c141890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:11.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.124+0000 7fb9545b5640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fb94c06ff20 con 0x7fb94c0744b0 2024-03-18T15:34:11.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.124+0000 7fb9545b5640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fb94c070170 con 0x7fb94c144390 2024-03-18T15:34:11.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.124+0000 7fb9545b5640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fb94c144d40 con 0x7fb94c141420 2024-03-18T15:34:11.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.124+0000 7fb952351640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb94c0744b0 0x7fb94c0b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:11.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.124+0000 7fb952351640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb94c0744b0 0x7fb94c0b53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45742/0 (socket says 172.21.15.5:45742) 2024-03-18T15:34:11.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.124+0000 7fb952351640 1 -- 172.21.15.5:0/2925756119 learned_addr learned my addr 172.21.15.5:0/2925756119 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:11.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.124+0000 7fb951b50640 1 --2- 172.21.15.5:0/2925756119 >> v2:172.21.15.80:3300/0 conn(0x7fb94c144390 0x7fb94c144800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:11.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.124+0000 7fb952b52640 1 --2- 172.21.15.5:0/2925756119 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb94c141420 0x7fb94c141890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:11.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.124+0000 7fb952351640 1 -- 172.21.15.5:0/2925756119 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb94c141420 msgr2=0x7fb94c141890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:11.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.124+0000 7fb952351640 1 --2- 172.21.15.5:0/2925756119 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb94c141420 0x7fb94c141890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:11.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.124+0000 7fb952351640 1 -- 172.21.15.5:0/2925756119 >> v2:172.21.15.80:3300/0 conn(0x7fb94c144390 msgr2=0x7fb94c144800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:11.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.124+0000 7fb952351640 1 --2- 172.21.15.5:0/2925756119 >> v2:172.21.15.80:3300/0 conn(0x7fb94c144390 0x7fb94c144800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:11.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.124+0000 7fb952351640 1 -- 172.21.15.5:0/2925756119 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb94c141dd0 con 0x7fb94c0744b0 2024-03-18T15:34:11.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.124+0000 7fb952b52640 1 --2- 172.21.15.5:0/2925756119 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb94c141420 0x7fb94c141890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:11.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.124+0000 7fb952351640 1 --2- 172.21.15.5:0/2925756119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb94c0744b0 0x7fb94c0b53f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb94c06e870 tx=0x7fb94801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ea30dac8b234527 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:11.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.124+0000 7fb95134f640 1 -- 172.21.15.5:0/2925756119 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9480ab650 con 0x7fb94c0744b0 2024-03-18T15:34:11.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.124+0000 7fb95134f640 1 -- 172.21.15.5:0/2925756119 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9480b0070 con 0x7fb94c0744b0 2024-03-18T15:34:11.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.124+0000 7fb95134f640 1 -- 172.21.15.5:0/2925756119 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9480aba10 con 0x7fb94c0744b0 2024-03-18T15:34:11.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.128+0000 7fb9545b5640 1 -- 172.21.15.5:0/2925756119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb94c0744b0 msgr2=0x7fb94c0b53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:11.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.128+0000 7fb9545b5640 1 --2- 172.21.15.5:0/2925756119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb94c0744b0 0x7fb94c0b53f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb94c06e870 tx=0x7fb94801c490 comp rx=0 tx=0).stop 2024-03-18T15:34:11.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.128+0000 7fb9545b5640 1 -- 172.21.15.5:0/2925756119 shutdown_connections 2024-03-18T15:34:11.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.128+0000 7fb9545b5640 1 --2- 172.21.15.5:0/2925756119 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb94c141420 0x7fb94c141890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:11.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.128+0000 7fb9545b5640 1 --2- 172.21.15.5:0/2925756119 >> v2:172.21.15.80:3300/0 conn(0x7fb94c144390 0x7fb94c144800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:11.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.128+0000 7fb9545b5640 1 --2- 172.21.15.5:0/2925756119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb94c0744b0 0x7fb94c0b53f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:11.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.128+0000 7fb9545b5640 1 -- 172.21.15.5:0/2925756119 >> 172.21.15.5:0/2925756119 conn(0x7fb94c0b9190 msgr2=0x7fb94c0b95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:11.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.128+0000 7fb9545b5640 1 -- 172.21.15.5:0/2925756119 shutdown_connections 2024-03-18T15:34:11.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.128+0000 7fb9545b5640 1 -- 172.21.15.5:0/2925756119 wait complete. 2024-03-18T15:34:11.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.128+0000 7fb9545b5640 1 Processor -- start 2024-03-18T15:34:11.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.128+0000 7fb9545b5640 1 -- start start 2024-03-18T15:34:11.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.128+0000 7fb9545b5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb94c0744b0 0x7fb94c1e2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:11.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.128+0000 7fb9545b5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb94c141420 0x7fb94c13b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:11.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.128+0000 7fb9545b5640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb94c144390 0x7fb94c13f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:11.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.128+0000 7fb9545b5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb94c0b4c20 con 0x7fb94c141420 2024-03-18T15:34:11.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.128+0000 7fb9545b5640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb94c0b2bd0 con 0x7fb94c0744b0 2024-03-18T15:34:11.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.128+0000 7fb9545b5640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fb94c147ef0 con 0x7fb94c144390 2024-03-18T15:34:11.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.128+0000 7fb951b50640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb94c141420 0x7fb94c13b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:11.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.128+0000 7fb951b50640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb94c141420 0x7fb94c13b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45754/0 (socket says 172.21.15.5:45754) 2024-03-18T15:34:11.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.128+0000 7fb951b50640 1 -- 172.21.15.5:0/3493966945 learned_addr learned my addr 172.21.15.5:0/3493966945 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:11.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.128+0000 7fb952351640 1 --2- 172.21.15.5:0/3493966945 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb94c0744b0 0x7fb94c1e2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:11.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.128+0000 7fb952b52640 1 --2- 172.21.15.5:0/3493966945 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb94c144390 0x7fb94c13f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:11.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.128+0000 7fb951b50640 1 -- 172.21.15.5:0/3493966945 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb94c144390 msgr2=0x7fb94c13f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:11.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.128+0000 7fb951b50640 1 --2- 172.21.15.5:0/3493966945 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb94c144390 0x7fb94c13f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:11.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.128+0000 7fb951b50640 1 -- 172.21.15.5:0/3493966945 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb94c0744b0 msgr2=0x7fb94c1e2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:11.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.128+0000 7fb951b50640 1 --2- 172.21.15.5:0/3493966945 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb94c0744b0 0x7fb94c1e2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:11.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.128+0000 7fb951b50640 1 -- 172.21.15.5:0/3493966945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb94c13b230 con 0x7fb94c141420 2024-03-18T15:34:11.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.128+0000 7fb951b50640 1 --2- 172.21.15.5:0/3493966945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb94c141420 0x7fb94c13b8f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb93c00d5f0 tx=0x7fb93c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:11.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.128+0000 7fb942ffd640 1 -- 172.21.15.5:0/3493966945 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93c0125f0 con 0x7fb94c141420 2024-03-18T15:34:11.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.128+0000 7fb942ffd640 1 -- 172.21.15.5:0/3493966945 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb93c019070 con 0x7fb94c141420 2024-03-18T15:34:11.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.128+0000 7fb942ffd640 1 -- 172.21.15.5:0/3493966945 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93c0129b0 con 0x7fb94c141420 2024-03-18T15:34:11.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.128+0000 7fb9545b5640 1 -- 172.21.15.5:0/3493966945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb94c13e670 con 0x7fb94c141420 2024-03-18T15:34:11.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.128+0000 7fb9545b5640 1 -- 172.21.15.5:0/3493966945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb94c14e720 con 0x7fb94c141420 2024-03-18T15:34:11.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.132+0000 7fb9545b5640 1 -- 172.21.15.5:0/3493966945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb94c13b430 con 0x7fb94c141420 2024-03-18T15:34:11.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.132+0000 7fb942ffd640 1 -- 172.21.15.5:0/3493966945 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fb93c012b50 con 0x7fb94c141420 2024-03-18T15:34:11.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.132+0000 7fb942ffd640 1 --2- 172.21.15.5:0/3493966945 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fb930067660 0x7fb930069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:11.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.132+0000 7fb952351640 1 --2- 172.21.15.5:0/3493966945 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fb930067660 0x7fb930069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:11.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.132+0000 7fb942ffd640 1 -- 172.21.15.5:0/3493966945 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb93c085ab0 con 0x7fb94c141420 2024-03-18T15:34:11.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.132+0000 7fb952351640 1 --2- 172.21.15.5:0/3493966945 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fb930067660 0x7fb930069b20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb9480af000 tx=0x7fb9480a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:11.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.136+0000 7fb942ffd640 1 -- 172.21.15.5:0/3493966945 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fb94c13b430 con 0x7fb94c141420 2024-03-18T15:34:11.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.292+0000 7fb9545b5640 1 -- 172.21.15.5:0/3493966945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb94c04f550 con 0x7fb94c141420 2024-03-18T15:34:11.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.292+0000 7fb942ffd640 1 -- 172.21.15.5:0/3493966945 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16449 (secure 0 0 0) 0x7fb93c0560d0 con 0x7fb94c141420 2024-03-18T15:34:11.296 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:34:11.296 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":21,"fsid":"ef54fe03-5aa7-4eb2-9b1e-66c9cc345b94","created":"2024-03-18T15:33:18.086614+0000","modified":"2024-03-18T15:33:43.984050+0000","last_up_change":"2024-03-18T15:33:29.229911+0000","last_in_change":"2024-03-18T15:33:20.323840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:28.963186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T15:33:40.533646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T15:33:41.323519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df1c754b-af59-4d16-8919-fbb6386b3c48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6804","nonce":976264754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6808","nonce":976264754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6815","nonce":976264754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6811","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6813","nonce":976264754}]},"public_addr":"172.21.15.5:6804/976264754","cluster_addr":"172.21.15.5:6808/976264754","heartbeat_back_addr":"172.21.15.5:6815/976264754","heartbeat_front_addr":"172.21.15.5:6813/976264754","state":["exists","up"]},{"osd":1,"uuid":"b9a08b5e-8af4-4144-bdba-d69ed1ac627a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6801","nonce":3901761126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6803","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6805","nonce":3901761126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6812","nonce":3901761126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6807","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6809","nonce":3901761126}]},"public_addr":"172.21.15.5:6801/3901761126","cluster_addr":"172.21.15.5:6805/3901761126","heartbeat_back_addr":"172.21.15.5:6812/3901761126","heartbeat_front_addr":"172.21.15.5:6809/3901761126","state":["exists","up"]},{"osd":2,"uuid":"c3e8bb76-e382-439d-b380-62ef92f1255a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6817","nonce":502968448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6819","nonce":502968448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6823","nonce":502968448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6821","nonce":502968448}]},"public_addr":"172.21.15.5:6817/502968448","cluster_addr":"172.21.15.5:6819/502968448","heartbeat_back_addr":"172.21.15.5:6823/502968448","heartbeat_front_addr":"172.21.15.5:6821/502968448","state":["exists","up"]},{"osd":3,"uuid":"e8c94625-f2cd-4eba-b53d-2e7356e1c02a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6825","nonce":2425932735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6827","nonce":2425932735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6831","nonce":2425932735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6829","nonce":2425932735}]},"public_addr":"172.21.15.5:6825/2425932735","cluster_addr":"172.21.15.5:6827/2425932735","heartbeat_back_addr":"172.21.15.5:6831/2425932735","heartbeat_front_addr":"172.21.15.5:6829/2425932735","state":["exists","up"]},{"osd":4,"uuid":"0099dd52-6265-4d32-82e4-3ef16e73add0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6817","nonce":976514285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6819","nonce":976514285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6823","nonce":976514285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6821","nonce":976514285}]},"public_addr":"172.21.15.80:6817/976514285","cluster_addr":"172.21.15.80:6819/976514285","heartbeat_back_addr":"172.21.15.80:6823/976514285","heartbeat_front_addr":"172.21.15.80:6821/976514285","state":["exists","up"]},{"osd":5,"uuid":"ed10c84f-8c39-41cc-9b64-7c482a94a48e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6825","nonce":627153447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6827","nonce":627153447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6831","nonce":627153447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6829","nonce":627153447}]},"public_addr":"172.21.15.80:6825/627153447","cluster_addr":"172.21.15.80:6827/627153447","heartbeat_back_addr":"172.21.15.80:6831/627153447","heartbeat_front_addr":"172.21.15.80:6829/627153447","state":["exists","up"]},{"osd":6,"uuid":"c7a3538a-7048-4fba-8781-b440f03c25db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6803","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6805","nonce":488342527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6807","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6809","nonce":488342527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6815","nonce":488342527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6811","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6813","nonce":488342527}]},"public_addr":"172.21.15.80:6805/488342527","cluster_addr":"172.21.15.80:6809/488342527","heartbeat_back_addr":"172.21.15.80:6815/488342527","heartbeat_front_addr":"172.21.15.80:6813/488342527","state":["exists","up"]},{"osd":7,"uuid":"f9418104-3a97-4110-bc3a-1f8151d0f757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6801","nonce":982831666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6804","nonce":982831666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6812","nonce":982831666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6808","nonce":982831666}]},"public_addr":"172.21.15.80:6801/982831666","cluster_addr":"172.21.15.80:6804/982831666","heartbeat_back_addr":"172.21.15.80:6812/982831666","heartbeat_front_addr":"172.21.15.80:6808/982831666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.970828+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.858116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.987556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.864603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.756293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.591502+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:34:11.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.296+0000 7fb9545b5640 1 -- 172.21.15.5:0/3493966945 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fb930067660 msgr2=0x7fb930069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:11.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.296+0000 7fb9545b5640 1 --2- 172.21.15.5:0/3493966945 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fb930067660 0x7fb930069b20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb9480af000 tx=0x7fb9480a7000 comp rx=0 tx=0).stop 2024-03-18T15:34:11.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.296+0000 7fb9545b5640 1 -- 172.21.15.5:0/3493966945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb94c141420 msgr2=0x7fb94c13b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:11.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.296+0000 7fb9545b5640 1 --2- 172.21.15.5:0/3493966945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb94c141420 0x7fb94c13b8f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb93c00d5f0 tx=0x7fb93c00dab0 comp rx=0 tx=0).stop 2024-03-18T15:34:11.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.296+0000 7fb9545b5640 1 -- 172.21.15.5:0/3493966945 shutdown_connections 2024-03-18T15:34:11.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.296+0000 7fb9545b5640 1 --2- 172.21.15.5:0/3493966945 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fb930067660 0x7fb930069b20 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:11.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.296+0000 7fb9545b5640 1 --2- 172.21.15.5:0/3493966945 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb94c144390 0x7fb94c13f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:11.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.296+0000 7fb9545b5640 1 --2- 172.21.15.5:0/3493966945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb94c141420 0x7fb94c13b8f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:11.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.296+0000 7fb9545b5640 1 --2- 172.21.15.5:0/3493966945 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb94c0744b0 0x7fb94c1e2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:11.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.296+0000 7fb9545b5640 1 -- 172.21.15.5:0/3493966945 >> 172.21.15.5:0/3493966945 conn(0x7fb94c0b9190 msgr2=0x7fb94c1408b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:11.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.296+0000 7fb9545b5640 1 -- 172.21.15.5:0/3493966945 shutdown_connections 2024-03-18T15:34:11.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.296+0000 7fb9545b5640 1 -- 172.21.15.5:0/3493966945 wait complete. 2024-03-18T15:34:11.314 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-18T15:34:11.314 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:34:11.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.428+0000 7fbd1dcb6640 1 Processor -- start 2024-03-18T15:34:11.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.428+0000 7fbd1dcb6640 1 -- start start 2024-03-18T15:34:11.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.432+0000 7fbd1dcb6640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbd180b5710 0x7fbd1814a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:11.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.432+0000 7fbd1dcb6640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbd1814ae60 0x7fbd18140ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:11.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.432+0000 7fbd1dcb6640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fbd18141430 0x7fbd180b7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:11.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.432+0000 7fbd1dcb6640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fbd180b4320 con 0x7fbd180b5710 2024-03-18T15:34:11.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.432+0000 7fbd1dcb6640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fbd180b4570 con 0x7fbd18141430 2024-03-18T15:34:11.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.432+0000 7fbd1dcb6640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fbd180ba0c0 con 0x7fbd1814ae60 2024-03-18T15:34:11.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.432+0000 7fbd177fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbd180b5710 0x7fbd1814a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:11.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.432+0000 7fbd177fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fbd180b5710 0x7fbd1814a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45770/0 (socket says 172.21.15.5:45770) 2024-03-18T15:34:11.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.432+0000 7fbd177fe640 1 -- 172.21.15.5:0/23464330 learned_addr learned my addr 172.21.15.5:0/23464330 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:11.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.432+0000 7fbd16ffd640 1 --2- 172.21.15.5:0/23464330 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbd1814ae60 0x7fbd18140ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:11.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.432+0000 7fbd17fff640 1 --2- 172.21.15.5:0/23464330 >> v2:172.21.15.80:3300/0 conn(0x7fbd18141430 0x7fbd180b7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:11.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.432+0000 7fbd177fe640 1 -- 172.21.15.5:0/23464330 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbd1814ae60 msgr2=0x7fbd18140ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:11.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.432+0000 7fbd177fe640 1 --2- 172.21.15.5:0/23464330 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbd1814ae60 0x7fbd18140ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:11.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.432+0000 7fbd177fe640 1 -- 172.21.15.5:0/23464330 >> v2:172.21.15.80:3300/0 conn(0x7fbd18141430 msgr2=0x7fbd180b7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:11.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.432+0000 7fbd177fe640 1 --2- 172.21.15.5:0/23464330 >> v2:172.21.15.80:3300/0 conn(0x7fbd18141430 0x7fbd180b7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:11.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.432+0000 7fbd177fe640 1 -- 172.21.15.5:0/23464330 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd180b9d50 con 0x7fbd180b5710 2024-03-18T15:34:11.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.432+0000 7fbd177fe640 1 --2- 172.21.15.5:0/23464330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd180b5710 0x7fbd1814a920 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbd08009870 tx=0x7fbd0801c140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=746d0430f0aa08d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:11.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.432+0000 7fbd167fc640 1 -- 172.21.15.5:0/23464330 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd080b0070 con 0x7fbd180b5710 2024-03-18T15:34:11.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.432+0000 7fbd167fc640 1 -- 172.21.15.5:0/23464330 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd080ab600 con 0x7fbd180b5710 2024-03-18T15:34:11.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.432+0000 7fbd167fc640 1 -- 172.21.15.5:0/23464330 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd080ab9d0 con 0x7fbd180b5710 2024-03-18T15:34:11.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.432+0000 7fbd1dcb6640 1 -- 172.21.15.5:0/23464330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd180b5710 msgr2=0x7fbd1814a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:11.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.432+0000 7fbd1dcb6640 1 --2- 172.21.15.5:0/23464330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd180b5710 0x7fbd1814a920 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbd08009870 tx=0x7fbd0801c140 comp rx=0 tx=0).stop 2024-03-18T15:34:11.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.432+0000 7fbd1dcb6640 1 -- 172.21.15.5:0/23464330 shutdown_connections 2024-03-18T15:34:11.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.432+0000 7fbd1dcb6640 1 --2- 172.21.15.5:0/23464330 >> v2:172.21.15.80:3300/0 conn(0x7fbd18141430 0x7fbd180b7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:11.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.432+0000 7fbd1dcb6640 1 --2- 172.21.15.5:0/23464330 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbd1814ae60 0x7fbd18140ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:11.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.432+0000 7fbd1dcb6640 1 --2- 172.21.15.5:0/23464330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd180b5710 0x7fbd1814a920 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:11.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.432+0000 7fbd1dcb6640 1 -- 172.21.15.5:0/23464330 >> 172.21.15.5:0/23464330 conn(0x7fbd180673c0 msgr2=0x7fbd180677d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:11.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.432+0000 7fbd1dcb6640 1 -- 172.21.15.5:0/23464330 shutdown_connections 2024-03-18T15:34:11.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.432+0000 7fbd1dcb6640 1 -- 172.21.15.5:0/23464330 wait complete. 2024-03-18T15:34:11.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.436+0000 7fbd1dcb6640 1 Processor -- start 2024-03-18T15:34:11.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.436+0000 7fbd1dcb6640 1 -- start start 2024-03-18T15:34:11.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.436+0000 7fbd1dcb6640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbd180b5710 0x7fbd181e0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:11.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.436+0000 7fbd1dcb6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd18141430 0x7fbd180baaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:11.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.436+0000 7fbd1dcb6640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbd1814ae60 0x7fbd18141f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:11.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.436+0000 7fbd1dcb6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbd180b50e0 con 0x7fbd18141430 2024-03-18T15:34:11.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.436+0000 7fbd1dcb6640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fbd180b5af0 con 0x7fbd180b5710 2024-03-18T15:34:11.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.436+0000 7fbd1dcb6640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fbd180b5260 con 0x7fbd1814ae60 2024-03-18T15:34:11.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.436+0000 7fbd16ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd18141430 0x7fbd180baaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:11.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.436+0000 7fbd16ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd18141430 0x7fbd180baaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45780/0 (socket says 172.21.15.5:45780) 2024-03-18T15:34:11.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.436+0000 7fbd16ffd640 1 -- 172.21.15.5:0/2257576716 learned_addr learned my addr 172.21.15.5:0/2257576716 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:11.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.436+0000 7fbd17fff640 1 --2- 172.21.15.5:0/2257576716 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbd1814ae60 0x7fbd18141f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:11.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.436+0000 7fbd177fe640 1 --2- 172.21.15.5:0/2257576716 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbd180b5710 0x7fbd181e0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:11.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.436+0000 7fbd16ffd640 1 -- 172.21.15.5:0/2257576716 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbd1814ae60 msgr2=0x7fbd18141f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:11.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.436+0000 7fbd16ffd640 1 --2- 172.21.15.5:0/2257576716 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbd1814ae60 0x7fbd18141f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:11.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.436+0000 7fbd16ffd640 1 -- 172.21.15.5:0/2257576716 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbd180b5710 msgr2=0x7fbd181e0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:11.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.436+0000 7fbd16ffd640 1 --2- 172.21.15.5:0/2257576716 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbd180b5710 0x7fbd181e0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:11.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.436+0000 7fbd16ffd640 1 -- 172.21.15.5:0/2257576716 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd18143210 con 0x7fbd18141430 2024-03-18T15:34:11.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.436+0000 7fbd16ffd640 1 --2- 172.21.15.5:0/2257576716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd18141430 0x7fbd180baaf0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c00d5f0 tx=0x7fbd0c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:11.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.436+0000 7fbcfffff640 1 -- 172.21.15.5:0/2257576716 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c0125f0 con 0x7fbd18141430 2024-03-18T15:34:11.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.436+0000 7fbcfffff640 1 -- 172.21.15.5:0/2257576716 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd0c019070 con 0x7fbd18141430 2024-03-18T15:34:11.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.436+0000 7fbd1dcb6640 1 -- 172.21.15.5:0/2257576716 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd181e0110 con 0x7fbd18141430 2024-03-18T15:34:11.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.436+0000 7fbcfffff640 1 -- 172.21.15.5:0/2257576716 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c0129b0 con 0x7fbd18141430 2024-03-18T15:34:11.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.436+0000 7fbd1dcb6640 1 -- 172.21.15.5:0/2257576716 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd18140300 con 0x7fbd18141430 2024-03-18T15:34:11.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.436+0000 7fbd1dcb6640 1 -- 172.21.15.5:0/2257576716 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd18140620 con 0x7fbd18141430 2024-03-18T15:34:11.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.440+0000 7fbcfffff640 1 -- 172.21.15.5:0/2257576716 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fbd0c012b50 con 0x7fbd18141430 2024-03-18T15:34:11.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.440+0000 7fbcfffff640 1 --2- 172.21.15.5:0/2257576716 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fbd04067690 0x7fbd04069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:11.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.440+0000 7fbcfffff640 1 -- 172.21.15.5:0/2257576716 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fbd0c085a30 con 0x7fbd18141430 2024-03-18T15:34:11.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.440+0000 7fbd177fe640 1 --2- 172.21.15.5:0/2257576716 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fbd04067690 0x7fbd04069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:11.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.440+0000 7fbd177fe640 1 --2- 172.21.15.5:0/2257576716 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fbd04067690 0x7fbd04069b50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbd080ad000 tx=0x7fbd080a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:11.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.444+0000 7fbcfffff640 1 -- 172.21.15.5:0/2257576716 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fbd0c0560f0 con 0x7fbd18141430 2024-03-18T15:34:11.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.636+0000 7fbd1dcb6640 1 -- 172.21.15.5:0/2257576716 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd1804f550 con 0x7fbd18141430 2024-03-18T15:34:11.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.636+0000 7fbcfffff640 1 -- 172.21.15.5:0/2257576716 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+3521 (secure 0 0 0) 0x7fbd0c05a190 con 0x7fbd18141430 2024-03-18T15:34:11.641 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:34:11.641 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/629589896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":629589896},{"type":"v1","addr":"172.21.15.5:6835","nonce":629589896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-18T15:33:42.338926+0000","modified":"2024-03-18T15:33:45.376646+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6833/1236603350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":1236603350},{"type":"v1","addr":"172.21.15.80:6833","nonce":1236603350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4272,"qdb_cluster":[4272]},"id":1}]} 2024-03-18T15:34:11.641 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 8 2024-03-18T15:34:11.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.640+0000 7fbd1dcb6640 1 -- 172.21.15.5:0/2257576716 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fbd04067690 msgr2=0x7fbd04069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:11.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.640+0000 7fbd1dcb6640 1 --2- 172.21.15.5:0/2257576716 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fbd04067690 0x7fbd04069b50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbd080ad000 tx=0x7fbd080a6000 comp rx=0 tx=0).stop 2024-03-18T15:34:11.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.640+0000 7fbd1dcb6640 1 -- 172.21.15.5:0/2257576716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd18141430 msgr2=0x7fbd180baaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:11.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.640+0000 7fbd1dcb6640 1 --2- 172.21.15.5:0/2257576716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd18141430 0x7fbd180baaf0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c00d5f0 tx=0x7fbd0c00dab0 comp rx=0 tx=0).stop 2024-03-18T15:34:11.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.640+0000 7fbd1dcb6640 1 -- 172.21.15.5:0/2257576716 shutdown_connections 2024-03-18T15:34:11.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.640+0000 7fbd1dcb6640 1 --2- 172.21.15.5:0/2257576716 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fbd04067690 0x7fbd04069b50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:11.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.640+0000 7fbd1dcb6640 1 --2- 172.21.15.5:0/2257576716 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fbd1814ae60 0x7fbd18141f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:11.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.640+0000 7fbd1dcb6640 1 --2- 172.21.15.5:0/2257576716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbd18141430 0x7fbd180baaf0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:11.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.640+0000 7fbd1dcb6640 1 --2- 172.21.15.5:0/2257576716 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbd180b5710 0x7fbd181e0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:11.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.640+0000 7fbd1dcb6640 1 -- 172.21.15.5:0/2257576716 >> 172.21.15.5:0/2257576716 conn(0x7fbd180673c0 msgr2=0x7fbd18136ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:11.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.640+0000 7fbd1dcb6640 1 -- 172.21.15.5:0/2257576716 shutdown_connections 2024-03-18T15:34:11.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.640+0000 7fbd1dcb6640 1 -- 172.21.15.5:0/2257576716 wait complete. 2024-03-18T15:34:11.659 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:34:11.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.780+0000 7fe495825640 1 Processor -- start 2024-03-18T15:34:11.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.780+0000 7fe495825640 1 -- start start 2024-03-18T15:34:11.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.780+0000 7fe495825640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe4900b31d0 0x7fe49013f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:11.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.780+0000 7fe495825640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe490149270 0x7fe49013f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:11.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.780+0000 7fe495825640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe490074930 con 0x7fe490074550 2024-03-18T15:34:11.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.780+0000 7fe495825640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fe490071020 con 0x7fe4900b31d0 2024-03-18T15:34:11.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.780+0000 7fe495825640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fe490071300 con 0x7fe490149270 2024-03-18T15:34:11.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.780+0000 7fe48f7fe640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe490149270 0x7fe49013f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:11.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.780+0000 7fe48e7fc640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe4900b31d0 0x7fe49013f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:11.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.780+0000 7fe48f7fe640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe490149270 0x7fe49013f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:54746/0 (socket says 172.21.15.5:54746) 2024-03-18T15:34:11.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.780+0000 7fe48e7fc640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe4900b31d0 0x7fe49013f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:48896/0 (socket says 172.21.15.5:48896) 2024-03-18T15:34:11.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.780+0000 7fe48f7fe640 1 -- 172.21.15.5:0/4093632017 learned_addr learned my addr 172.21.15.5:0/4093632017 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:11.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.780+0000 7fe48e7fc640 1 -- 172.21.15.5:0/4093632017 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe490149270 msgr2=0x7fe49013f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:11.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.780+0000 7fe48e7fc640 1 --2- 172.21.15.5:0/4093632017 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe490149270 0x7fe49013f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:11.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.780+0000 7fe48e7fc640 1 -- 172.21.15.5:0/4093632017 >> v1:172.21.15.80:6789/0 conn(0x7fe490074550 legacy=0x7fe490148b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:11.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.780+0000 7fe48e7fc640 1 -- 172.21.15.5:0/4093632017 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe49006fc40 con 0x7fe4900b31d0 2024-03-18T15:34:11.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.784+0000 7fe48e7fc640 1 --2- 172.21.15.5:0/4093632017 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4900b31d0 0x7fe49013f020 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe4840097c0 tx=0x7fe48401c380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b73f7896f310e9ce server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.784+0000 7fe48dffb640 1 -- 172.21.15.5:0/4093632017 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4840b0070 con 0x7fe4900b31d0 2024-03-18T15:34:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.784+0000 7fe48dffb640 1 -- 172.21.15.5:0/4093632017 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4840ab600 con 0x7fe4900b31d0 2024-03-18T15:34:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.784+0000 7fe48dffb640 1 -- 172.21.15.5:0/4093632017 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4840ab9d0 con 0x7fe4900b31d0 2024-03-18T15:34:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.784+0000 7fe495825640 1 -- 172.21.15.5:0/4093632017 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4900b31d0 msgr2=0x7fe49013f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.784+0000 7fe495825640 1 --2- 172.21.15.5:0/4093632017 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4900b31d0 0x7fe49013f020 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe4840097c0 tx=0x7fe48401c380 comp rx=0 tx=0).stop 2024-03-18T15:34:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.784+0000 7fe495825640 1 -- 172.21.15.5:0/4093632017 shutdown_connections 2024-03-18T15:34:11.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.784+0000 7fe495825640 1 --2- 172.21.15.5:0/4093632017 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe490149270 0x7fe49013f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:11.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.784+0000 7fe495825640 1 --2- 172.21.15.5:0/4093632017 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4900b31d0 0x7fe49013f020 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:11.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.784+0000 7fe495825640 1 -- 172.21.15.5:0/4093632017 >> 172.21.15.5:0/4093632017 conn(0x7fe4900b5d00 msgr2=0x7fe4900b6110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:11.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.784+0000 7fe495825640 1 -- 172.21.15.5:0/4093632017 shutdown_connections 2024-03-18T15:34:11.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.784+0000 7fe495825640 1 -- 172.21.15.5:0/4093632017 wait complete. 2024-03-18T15:34:11.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.784+0000 7fe495825640 1 Processor -- start 2024-03-18T15:34:11.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.784+0000 7fe495825640 1 -- start start 2024-03-18T15:34:11.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.784+0000 7fe495825640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe490074550 0x7fe4901dfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:11.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.784+0000 7fe495825640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe4900b31d0 0x7fe490138350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:11.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.784+0000 7fe495825640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe490149270 0x7fe4900b1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:11.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.784+0000 7fe495825640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe490152730 con 0x7fe490149270 2024-03-18T15:34:11.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.784+0000 7fe495825640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe4901525b0 con 0x7fe490074550 2024-03-18T15:34:11.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.784+0000 7fe495825640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fe4901528b0 con 0x7fe4900b31d0 2024-03-18T15:34:11.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.784+0000 7fe48f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe490149270 0x7fe4900b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:11.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.784+0000 7fe48f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe490149270 0x7fe4900b1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45794/0 (socket says 172.21.15.5:45794) 2024-03-18T15:34:11.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.784+0000 7fe48f7fe640 1 -- 172.21.15.5:0/2479957816 learned_addr learned my addr 172.21.15.5:0/2479957816 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:11.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.788+0000 7fe48e7fc640 1 --2- 172.21.15.5:0/2479957816 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe4900b31d0 0x7fe490138350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:11.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.788+0000 7fe48effd640 1 --2- 172.21.15.5:0/2479957816 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe490074550 0x7fe4901dfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:11.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.788+0000 7fe48f7fe640 1 -- 172.21.15.5:0/2479957816 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe4900b31d0 msgr2=0x7fe490138350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:11.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.788+0000 7fe48f7fe640 1 --2- 172.21.15.5:0/2479957816 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe4900b31d0 0x7fe490138350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:11.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.788+0000 7fe48f7fe640 1 -- 172.21.15.5:0/2479957816 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe490074550 msgr2=0x7fe4901dfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:11.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.788+0000 7fe48f7fe640 1 --2- 172.21.15.5:0/2479957816 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe490074550 0x7fe4901dfbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:11.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.788+0000 7fe48f7fe640 1 -- 172.21.15.5:0/2479957816 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4900b1c70 con 0x7fe490149270 2024-03-18T15:34:11.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.788+0000 7fe48f7fe640 1 --2- 172.21.15.5:0/2479957816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe490149270 0x7fe4900b1580 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe48800efc0 tx=0x7fe48800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:11.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.788+0000 7fe477fff640 1 -- 172.21.15.5:0/2479957816 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48800cd20 con 0x7fe490149270 2024-03-18T15:34:11.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.788+0000 7fe477fff640 1 -- 172.21.15.5:0/2479957816 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4880039a0 con 0x7fe490149270 2024-03-18T15:34:11.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.788+0000 7fe495825640 1 -- 172.21.15.5:0/2479957816 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4900b1ee0 con 0x7fe490149270 2024-03-18T15:34:11.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.788+0000 7fe477fff640 1 -- 172.21.15.5:0/2479957816 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe488003d90 con 0x7fe490149270 2024-03-18T15:34:11.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.788+0000 7fe495825640 1 -- 172.21.15.5:0/2479957816 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe490142b50 con 0x7fe490149270 2024-03-18T15:34:11.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.788+0000 7fe495825640 1 -- 172.21.15.5:0/2479957816 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe49004f550 con 0x7fe490149270 2024-03-18T15:34:11.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.788+0000 7fe477fff640 1 -- 172.21.15.5:0/2479957816 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fe488012050 con 0x7fe490149270 2024-03-18T15:34:11.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.788+0000 7fe477fff640 1 --2- 172.21.15.5:0/2479957816 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe47c067660 0x7fe47c069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:11.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.788+0000 7fe477fff640 1 -- 172.21.15.5:0/2479957816 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe488085be0 con 0x7fe490149270 2024-03-18T15:34:11.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.788+0000 7fe48effd640 1 --2- 172.21.15.5:0/2479957816 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe47c067660 0x7fe47c069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:11.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.788+0000 7fe48effd640 1 --2- 172.21.15.5:0/2479957816 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe47c067660 0x7fe47c069b20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe480007fd0 tx=0x7fe480009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:11.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.792+0000 7fe477fff640 1 -- 172.21.15.5:0/2479957816 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fe488010040 con 0x7fe490149270 2024-03-18T15:34:11.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.996+0000 7fe495825640 1 -- 172.21.15.5:0/2479957816 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe490149700 con 0x7fe490149270 2024-03-18T15:34:11.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.996+0000 7fe477fff640 1 -- 172.21.15.5:0/2479957816 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+3521 (secure 0 0 0) 0x7fe488056220 con 0x7fe490149270 2024-03-18T15:34:11.999 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:34:11.999 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/629589896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":629589896},{"type":"v1","addr":"172.21.15.5:6835","nonce":629589896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-18T15:33:42.338926+0000","modified":"2024-03-18T15:33:45.376646+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6833/1236603350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":1236603350},{"type":"v1","addr":"172.21.15.80:6833","nonce":1236603350}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4272,"qdb_cluster":[4272]},"id":1}]} 2024-03-18T15:34:11.999 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 8 2024-03-18T15:34:12.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.996+0000 7fe495825640 1 -- 172.21.15.5:0/2479957816 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe47c067660 msgr2=0x7fe47c069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:12.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.996+0000 7fe495825640 1 --2- 172.21.15.5:0/2479957816 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe47c067660 0x7fe47c069b20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe480007fd0 tx=0x7fe480009040 comp rx=0 tx=0).stop 2024-03-18T15:34:12.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.996+0000 7fe495825640 1 -- 172.21.15.5:0/2479957816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe490149270 msgr2=0x7fe4900b1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:12.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:11.996+0000 7fe495825640 1 --2- 172.21.15.5:0/2479957816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe490149270 0x7fe4900b1580 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe48800efc0 tx=0x7fe48800c490 comp rx=0 tx=0).stop 2024-03-18T15:34:12.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.000+0000 7fe495825640 1 -- 172.21.15.5:0/2479957816 shutdown_connections 2024-03-18T15:34:12.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.000+0000 7fe495825640 1 --2- 172.21.15.5:0/2479957816 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe47c067660 0x7fe47c069b20 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:12.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.000+0000 7fe495825640 1 --2- 172.21.15.5:0/2479957816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe490149270 0x7fe4900b1580 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:12.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.000+0000 7fe495825640 1 --2- 172.21.15.5:0/2479957816 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe4900b31d0 0x7fe490138350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:12.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.000+0000 7fe495825640 1 --2- 172.21.15.5:0/2479957816 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe490074550 0x7fe4901dfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:12.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.000+0000 7fe495825640 1 -- 172.21.15.5:0/2479957816 >> 172.21.15.5:0/2479957816 conn(0x7fe4900b5d00 msgr2=0x7fe4900b3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:12.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.000+0000 7fe495825640 1 -- 172.21.15.5:0/2479957816 shutdown_connections 2024-03-18T15:34:12.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.000+0000 7fe495825640 1 -- 172.21.15.5:0/2479957816 wait complete. 2024-03-18T15:34:12.018 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:34:12.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.132+0000 7f977ad58640 1 Processor -- start 2024-03-18T15:34:12.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.132+0000 7f977ad58640 1 -- start start 2024-03-18T15:34:12.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.132+0000 7f977ad58640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f977413f260 0x7f977413f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:12.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.132+0000 7f977ad58640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97740b2ab0 con 0x7f977406d2d0 2024-03-18T15:34:12.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.132+0000 7f977ad58640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97740b0ea0 con 0x7f977414dc70 2024-03-18T15:34:12.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.132+0000 7f977ad58640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f97740728c0 con 0x7f977413f260 2024-03-18T15:34:12.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.132+0000 7f9778af4640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f977406d2d0 0x7f97740b23a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51524/0 (socket says 172.21.15.5:51524) 2024-03-18T15:34:12.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.132+0000 7f9778af4640 1 -- 172.21.15.5:0/1062616352 learned_addr learned my addr 172.21.15.5:0/1062616352 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:12.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.132+0000 7f9773fff640 1 --2- 172.21.15.5:0/1062616352 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f977413f260 0x7f977413f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:12.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.132+0000 7f97737fe640 1 -- 172.21.15.5:0/1062616352 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3762103795 0 0) 0x7f97740b2ab0 con 0x7f977406d2d0 2024-03-18T15:34:12.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.132+0000 7f97737fe640 1 -- 172.21.15.5:0/1062616352 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f975c003530 con 0x7f977406d2d0 2024-03-18T15:34:12.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.132+0000 7f97737fe640 1 -- 172.21.15.5:0/1062616352 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 805793110 0 0) 0x7f975c003530 con 0x7f977406d2d0 2024-03-18T15:34:12.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.132+0000 7f97737fe640 1 -- 172.21.15.5:0/1062616352 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f977413f260 msgr2=0x7f977413f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:12.137 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.132+0000 7f97737fe640 1 --2- 172.21.15.5:0/1062616352 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f977413f260 0x7f977413f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:12.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.132+0000 7f97737fe640 1 -- 172.21.15.5:0/1062616352 >> v1:172.21.15.80:6789/0 conn(0x7f977414dc70 legacy=0x7f9774150060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:12.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.132+0000 7f97737fe640 1 -- 172.21.15.5:0/1062616352 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f977406d6b0 con 0x7f977406d2d0 2024-03-18T15:34:12.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.132+0000 7f97737fe640 1 -- 172.21.15.5:0/1062616352 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3837549003 0 0) 0x7f9764001680 con 0x7f977406d2d0 2024-03-18T15:34:12.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.132+0000 7f97737fe640 1 -- 172.21.15.5:0/1062616352 >> v1:172.21.15.5:6789/0 conn(0x7f977406d2d0 legacy=0x7f97740b23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:12.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.132+0000 7f97737fe640 1 --2- 172.21.15.5:0/1062616352 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f975c003ad0 0x7f975c003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:12.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.132+0000 7f97737fe640 1 --2- 172.21.15.5:0/1062616352 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f977413f260 0x7f975c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:12.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.132+0000 7f97737fe640 1 --2- 172.21.15.5:0/1062616352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f975c004da0 0x7f975c005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:12.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.132+0000 7f97737fe640 1 -- 172.21.15.5:0/1062616352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f97740728c0 con 0x7f975c004da0 2024-03-18T15:34:12.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.132+0000 7f97737fe640 1 -- 172.21.15.5:0/1062616352 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f975c00bc30 con 0x7f975c003ad0 2024-03-18T15:34:12.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.132+0000 7f97737fe640 1 -- 172.21.15.5:0/1062616352 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f975c00be30 con 0x7f977413f260 2024-03-18T15:34:12.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.132+0000 7f97792f5640 1 --2- 172.21.15.5:0/1062616352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f975c004da0 0x7f975c005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:12.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.132+0000 7f9778af4640 1 --2- 172.21.15.5:0/1062616352 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f975c003ad0 0x7f975c003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:12.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.132+0000 7f9773fff640 1 --2- 172.21.15.5:0/1062616352 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f977413f260 0x7f975c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:12.138 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.132+0000 7f97792f5640 1 -- 172.21.15.5:0/1062616352 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f977413f260 msgr2=0x7f975c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:12.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.132+0000 7f97792f5640 1 --2- 172.21.15.5:0/1062616352 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f977413f260 0x7f975c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:12.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.132+0000 7f97792f5640 1 -- 172.21.15.5:0/1062616352 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f975c003ad0 msgr2=0x7f975c003ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:12.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.132+0000 7f97792f5640 1 --2- 172.21.15.5:0/1062616352 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f975c003ad0 0x7f975c003ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:12.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.132+0000 7f97792f5640 1 -- 172.21.15.5:0/1062616352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f975c003530 con 0x7f975c004da0 2024-03-18T15:34:12.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.136+0000 7f97792f5640 1 --2- 172.21.15.5:0/1062616352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f975c004da0 0x7f975c005220 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f977406f250 tx=0x7f976c01c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5e0485c37deb5ee server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:12.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.136+0000 7f97737fe640 1 -- 172.21.15.5:0/1062616352 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976c001a90 con 0x7f975c004da0 2024-03-18T15:34:12.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.136+0000 7f97737fe640 1 -- 172.21.15.5:0/1062616352 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f976c003b80 con 0x7f975c004da0 2024-03-18T15:34:12.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.136+0000 7f97737fe640 1 -- 172.21.15.5:0/1062616352 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976c005080 con 0x7f975c004da0 2024-03-18T15:34:12.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.136+0000 7f977ad58640 1 -- 172.21.15.5:0/1062616352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f975c004da0 msgr2=0x7f975c005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:12.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.136+0000 7f977ad58640 1 --2- 172.21.15.5:0/1062616352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f975c004da0 0x7f975c005220 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f977406f250 tx=0x7f976c01c720 comp rx=0 tx=0).stop 2024-03-18T15:34:12.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.136+0000 7f97792f5640 1 -- 172.21.15.5:0/1062616352 reap_dead start 2024-03-18T15:34:12.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.136+0000 7f977ad58640 1 -- 172.21.15.5:0/1062616352 shutdown_connections 2024-03-18T15:34:12.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.136+0000 7f977ad58640 1 -- 172.21.15.5:0/1062616352 >> 172.21.15.5:0/1062616352 conn(0x7f97740b5670 msgr2=0x7f97740b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:12.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.136+0000 7f977ad58640 1 -- 172.21.15.5:0/1062616352 shutdown_connections 2024-03-18T15:34:12.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.136+0000 7f977ad58640 1 -- 172.21.15.5:0/1062616352 wait complete. 2024-03-18T15:34:12.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.136+0000 7f977ad58640 1 Processor -- start 2024-03-18T15:34:12.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.136+0000 7f977ad58640 1 -- start start 2024-03-18T15:34:12.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.136+0000 7f977ad58640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f975c004da0 0x7f9774144e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:12.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.136+0000 7f977ad58640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97741dec30 0x7f97741422d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:12.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.136+0000 7f977ad58640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97741df0b0 0x7f9774142810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:12.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.136+0000 7f977ad58640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f97740b0070 con 0x7f97741df0b0 2024-03-18T15:34:12.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.136+0000 7f977ad58640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f97740b1c30 con 0x7f97741dec30 2024-03-18T15:34:12.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.136+0000 7f977ad58640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f97740b01f0 con 0x7f975c004da0 2024-03-18T15:34:12.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.136+0000 7f97792f5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97741df0b0 0x7f9774142810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:12.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.136+0000 7f97792f5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97741df0b0 0x7f9774142810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45820/0 (socket says 172.21.15.5:45820) 2024-03-18T15:34:12.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.136+0000 7f97792f5640 1 -- 172.21.15.5:0/2672853136 learned_addr learned my addr 172.21.15.5:0/2672853136 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:12.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.140+0000 7f9773fff640 1 --2- 172.21.15.5:0/2672853136 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97741dec30 0x7f97741422d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:12.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.140+0000 7f9778af4640 1 --2- 172.21.15.5:0/2672853136 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f975c004da0 0x7f9774144e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:12.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.140+0000 7f97792f5640 1 -- 172.21.15.5:0/2672853136 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f975c004da0 msgr2=0x7f9774144e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:12.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.140+0000 7f97792f5640 1 --2- 172.21.15.5:0/2672853136 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f975c004da0 0x7f9774144e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:12.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.140+0000 7f97792f5640 1 -- 172.21.15.5:0/2672853136 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97741dec30 msgr2=0x7f97741422d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:12.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.140+0000 7f97792f5640 1 --2- 172.21.15.5:0/2672853136 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97741dec30 0x7f97741422d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:12.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.140+0000 7f97792f5640 1 -- 172.21.15.5:0/2672853136 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9774140620 con 0x7f97741df0b0 2024-03-18T15:34:12.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.140+0000 7f9773fff640 1 --2- 172.21.15.5:0/2672853136 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97741dec30 0x7f97741422d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:34:12.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.140+0000 7f97792f5640 1 --2- 172.21.15.5:0/2672853136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97741df0b0 0x7f9774142810 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f976c01cc90 tx=0x7f975c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:12.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.140+0000 7f97717fa640 1 -- 172.21.15.5:0/2672853136 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976c0bd070 con 0x7f97741df0b0 2024-03-18T15:34:12.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.140+0000 7f97717fa640 1 -- 172.21.15.5:0/2672853136 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f976c005640 con 0x7f97741df0b0 2024-03-18T15:34:12.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.140+0000 7f977ad58640 1 -- 172.21.15.5:0/2672853136 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9774140b70 con 0x7f97741df0b0 2024-03-18T15:34:12.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.140+0000 7f97717fa640 1 -- 172.21.15.5:0/2672853136 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976c001a90 con 0x7f97741df0b0 2024-03-18T15:34:12.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.140+0000 7f977ad58640 1 -- 172.21.15.5:0/2672853136 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9774140890 con 0x7f97741df0b0 2024-03-18T15:34:12.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.140+0000 7f977ad58640 1 -- 172.21.15.5:0/2672853136 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f977404f550 con 0x7f97741df0b0 2024-03-18T15:34:12.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.140+0000 7f97717fa640 1 -- 172.21.15.5:0/2672853136 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f976c0b1040 con 0x7f97741df0b0 2024-03-18T15:34:12.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.140+0000 7f97717fa640 1 --2- 172.21.15.5:0/2672853136 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f9760067660 0x7f9760069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:12.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.140+0000 7f97717fa640 1 -- 172.21.15.5:0/2672853136 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f976c11ef90 con 0x7f97741df0b0 2024-03-18T15:34:12.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.140+0000 7f9778af4640 1 --2- 172.21.15.5:0/2672853136 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f9760067660 0x7f9760069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:12.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.140+0000 7f9778af4640 1 --2- 172.21.15.5:0/2672853136 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f9760067660 0x7f9760069b20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9764001060 tx=0x7f9764009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:12.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.144+0000 7f97717fa640 1 -- 172.21.15.5:0/2672853136 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f976c0ad020 con 0x7f97741df0b0 2024-03-18T15:34:12.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.308+0000 7f977ad58640 1 -- 172.21.15.5:0/2672853136 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9774142ff0 con 0x7f97741df0b0 2024-03-18T15:34:12.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.308+0000 7f97717fa640 1 -- 172.21.15.5:0/2672853136 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16449 (secure 0 0 0) 0x7f976c016430 con 0x7f97741df0b0 2024-03-18T15:34:12.313 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:34:12.313 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":21,"fsid":"ef54fe03-5aa7-4eb2-9b1e-66c9cc345b94","created":"2024-03-18T15:33:18.086614+0000","modified":"2024-03-18T15:33:43.984050+0000","last_up_change":"2024-03-18T15:33:29.229911+0000","last_in_change":"2024-03-18T15:33:20.323840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:28.963186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-18T15:33:40.533646+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-18T15:33:41.323519+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df1c754b-af59-4d16-8919-fbb6386b3c48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6804","nonce":976264754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6808","nonce":976264754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6815","nonce":976264754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6811","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6813","nonce":976264754}]},"public_addr":"172.21.15.5:6804/976264754","cluster_addr":"172.21.15.5:6808/976264754","heartbeat_back_addr":"172.21.15.5:6815/976264754","heartbeat_front_addr":"172.21.15.5:6813/976264754","state":["exists","up"]},{"osd":1,"uuid":"b9a08b5e-8af4-4144-bdba-d69ed1ac627a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6801","nonce":3901761126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6803","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6805","nonce":3901761126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6812","nonce":3901761126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6807","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6809","nonce":3901761126}]},"public_addr":"172.21.15.5:6801/3901761126","cluster_addr":"172.21.15.5:6805/3901761126","heartbeat_back_addr":"172.21.15.5:6812/3901761126","heartbeat_front_addr":"172.21.15.5:6809/3901761126","state":["exists","up"]},{"osd":2,"uuid":"c3e8bb76-e382-439d-b380-62ef92f1255a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6817","nonce":502968448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6819","nonce":502968448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6823","nonce":502968448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6821","nonce":502968448}]},"public_addr":"172.21.15.5:6817/502968448","cluster_addr":"172.21.15.5:6819/502968448","heartbeat_back_addr":"172.21.15.5:6823/502968448","heartbeat_front_addr":"172.21.15.5:6821/502968448","state":["exists","up"]},{"osd":3,"uuid":"e8c94625-f2cd-4eba-b53d-2e7356e1c02a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6825","nonce":2425932735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6827","nonce":2425932735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6831","nonce":2425932735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6829","nonce":2425932735}]},"public_addr":"172.21.15.5:6825/2425932735","cluster_addr":"172.21.15.5:6827/2425932735","heartbeat_back_addr":"172.21.15.5:6831/2425932735","heartbeat_front_addr":"172.21.15.5:6829/2425932735","state":["exists","up"]},{"osd":4,"uuid":"0099dd52-6265-4d32-82e4-3ef16e73add0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6817","nonce":976514285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6819","nonce":976514285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6823","nonce":976514285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6821","nonce":976514285}]},"public_addr":"172.21.15.80:6817/976514285","cluster_addr":"172.21.15.80:6819/976514285","heartbeat_back_addr":"172.21.15.80:6823/976514285","heartbeat_front_addr":"172.21.15.80:6821/976514285","state":["exists","up"]},{"osd":5,"uuid":"ed10c84f-8c39-41cc-9b64-7c482a94a48e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6825","nonce":627153447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6827","nonce":627153447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6831","nonce":627153447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6829","nonce":627153447}]},"public_addr":"172.21.15.80:6825/627153447","cluster_addr":"172.21.15.80:6827/627153447","heartbeat_back_addr":"172.21.15.80:6831/627153447","heartbeat_front_addr":"172.21.15.80:6829/627153447","state":["exists","up"]},{"osd":6,"uuid":"c7a3538a-7048-4fba-8781-b440f03c25db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6803","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6805","nonce":488342527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6807","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6809","nonce":488342527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6815","nonce":488342527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6811","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6813","nonce":488342527}]},"public_addr":"172.21.15.80:6805/488342527","cluster_addr":"172.21.15.80:6809/488342527","heartbeat_back_addr":"172.21.15.80:6815/488342527","heartbeat_front_addr":"172.21.15.80:6813/488342527","state":["exists","up"]},{"osd":7,"uuid":"f9418104-3a97-4110-bc3a-1f8151d0f757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6801","nonce":982831666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6804","nonce":982831666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6812","nonce":982831666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6808","nonce":982831666}]},"public_addr":"172.21.15.80:6801/982831666","cluster_addr":"172.21.15.80:6804/982831666","heartbeat_back_addr":"172.21.15.80:6812/982831666","heartbeat_front_addr":"172.21.15.80:6808/982831666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.970828+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.858116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.987556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.864603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.756293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.591502+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:34:12.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.312+0000 7f977ad58640 1 -- 172.21.15.5:0/2672853136 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f9760067660 msgr2=0x7f9760069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:12.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.312+0000 7f977ad58640 1 --2- 172.21.15.5:0/2672853136 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f9760067660 0x7f9760069b20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9764001060 tx=0x7f9764009040 comp rx=0 tx=0).stop 2024-03-18T15:34:12.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.312+0000 7f977ad58640 1 -- 172.21.15.5:0/2672853136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97741df0b0 msgr2=0x7f9774142810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:12.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.312+0000 7f977ad58640 1 --2- 172.21.15.5:0/2672853136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97741df0b0 0x7f9774142810 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f976c01cc90 tx=0x7f975c005840 comp rx=0 tx=0).stop 2024-03-18T15:34:12.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.312+0000 7f977ad58640 1 -- 172.21.15.5:0/2672853136 shutdown_connections 2024-03-18T15:34:12.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.312+0000 7f977ad58640 1 --2- 172.21.15.5:0/2672853136 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f9760067660 0x7f9760069b20 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:12.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.312+0000 7f977ad58640 1 --2- 172.21.15.5:0/2672853136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97741df0b0 0x7f9774142810 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:12.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.312+0000 7f977ad58640 1 --2- 172.21.15.5:0/2672853136 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97741dec30 0x7f97741422d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:12.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.312+0000 7f977ad58640 1 --2- 172.21.15.5:0/2672853136 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f975c004da0 0x7f9774144e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:12.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.312+0000 7f977ad58640 1 -- 172.21.15.5:0/2672853136 >> 172.21.15.5:0/2672853136 conn(0x7f97740b5670 msgr2=0x7f97740b4b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:12.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.312+0000 7f977ad58640 1 -- 172.21.15.5:0/2672853136 shutdown_connections 2024-03-18T15:34:12.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.312+0000 7f977ad58640 1 -- 172.21.15.5:0/2672853136 wait complete. 2024-03-18T15:34:12.332 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-18T15:34:12.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.448+0000 7f09aab79640 1 Processor -- start 2024-03-18T15:34:12.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.448+0000 7f09aab79640 1 -- start start 2024-03-18T15:34:12.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.448+0000 7f09aab79640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f09a40b6ad0 0x7f09a40b6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:12.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.448+0000 7f09aab79640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09a406b4e0 con 0x7f09a40b4580 2024-03-18T15:34:12.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.448+0000 7f09aab79640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09a40b5ec0 con 0x7f09a4148c50 2024-03-18T15:34:12.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.448+0000 7f09aab79640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f09a40bbb70 con 0x7f09a40b6ad0 2024-03-18T15:34:12.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.448+0000 7f09a3fff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f09a4148c50 0x7f09a41490b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.5:38040/0 (socket says 172.21.15.5:38040) 2024-03-18T15:34:12.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.448+0000 7f09a3fff640 1 -- 172.21.15.5:0/3558597877 learned_addr learned my addr 172.21.15.5:0/3558597877 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:12.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.448+0000 7f09a8915640 1 --2- 172.21.15.5:0/3558597877 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f09a40b6ad0 0x7f09a40b6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:12.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.448+0000 7f09a37fe640 1 -- 172.21.15.5:0/3558597877 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 353950499 0 0) 0x7f09a406b4e0 con 0x7f09a40b4580 2024-03-18T15:34:12.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.448+0000 7f09a37fe640 1 -- 172.21.15.5:0/3558597877 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f098c003530 con 0x7f09a40b4580 2024-03-18T15:34:12.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.448+0000 7f09a37fe640 1 -- 172.21.15.5:0/3558597877 <== mon.1 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2963467887 0 0) 0x7f09a40b5ec0 con 0x7f09a4148c50 2024-03-18T15:34:12.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.448+0000 7f09a37fe640 1 -- 172.21.15.5:0/3558597877 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f09a406b4e0 con 0x7f09a4148c50 2024-03-18T15:34:12.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.448+0000 7f09a8915640 1 -- 172.21.15.5:0/3558597877 >> v1:172.21.15.80:6789/0 conn(0x7f09a4148c50 legacy=0x7f09a41490b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:12.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.448+0000 7f09a37fe640 1 -- 172.21.15.5:0/3558597877 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3013669757 0 0) 0x7f098c003530 con 0x7f09a40b4580 2024-03-18T15:34:12.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.448+0000 7f09a8915640 1 -- 172.21.15.5:0/3558597877 >> v1:172.21.15.5:6789/0 conn(0x7f09a40b4580 legacy=0x7f09a40b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:12.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.448+0000 7f09a8915640 1 -- 172.21.15.5:0/3558597877 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09a40b1d50 con 0x7f09a40b6ad0 2024-03-18T15:34:12.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.452+0000 7f09a8915640 1 --2- 172.21.15.5:0/3558597877 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f09a40b6ad0 0x7f09a40b6eb0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f099c009dc0 tx=0x7f099c01c790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92e0d769033b618e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:12.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.452+0000 7f09a37fe640 1 -- 172.21.15.5:0/3558597877 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0994001680 con 0x7f09a40b6ad0 2024-03-18T15:34:12.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.452+0000 7f09a37fe640 1 -- 172.21.15.5:0/3558597877 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f099c0a8440 con 0x7f09a40b6ad0 2024-03-18T15:34:12.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.452+0000 7f09a37fe640 1 -- 172.21.15.5:0/3558597877 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099c0a87f0 con 0x7f09a40b6ad0 2024-03-18T15:34:12.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.452+0000 7f09aab79640 1 -- 172.21.15.5:0/3558597877 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f09a40b6ad0 msgr2=0x7f09a40b6eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:12.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.452+0000 7f09aab79640 1 --2- 172.21.15.5:0/3558597877 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f09a40b6ad0 0x7f09a40b6eb0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f099c009dc0 tx=0x7f099c01c790 comp rx=0 tx=0).stop 2024-03-18T15:34:12.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.452+0000 7f09aab79640 1 -- 172.21.15.5:0/3558597877 shutdown_connections 2024-03-18T15:34:12.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.452+0000 7f09aab79640 1 --2- 172.21.15.5:0/3558597877 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f09a40b6ad0 0x7f09a40b6eb0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:12.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.452+0000 7f09aab79640 1 -- 172.21.15.5:0/3558597877 >> 172.21.15.5:0/3558597877 conn(0x7f09a40bacb0 msgr2=0x7f09a40bb0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:12.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.452+0000 7f09aab79640 1 -- 172.21.15.5:0/3558597877 shutdown_connections 2024-03-18T15:34:12.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.452+0000 7f09aab79640 1 -- 172.21.15.5:0/3558597877 wait complete. 2024-03-18T15:34:12.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.452+0000 7f09aab79640 1 Processor -- start 2024-03-18T15:34:12.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.452+0000 7f09aab79640 1 -- start start 2024-03-18T15:34:12.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.456+0000 7f09aab79640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f09a40b4580 0x7f09a413e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:12.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.456+0000 7f09aab79640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09a40b6ad0 0x7f09a413b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:12.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.456+0000 7f09aab79640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09a4148c50 0x7f09a413b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:12.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.456+0000 7f09aab79640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f09a4153cd0 con 0x7f09a4148c50 2024-03-18T15:34:12.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.456+0000 7f09aab79640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f09a40b7600 con 0x7f09a40b6ad0 2024-03-18T15:34:12.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.456+0000 7f09aab79640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f09a4153e50 con 0x7f09a40b4580 2024-03-18T15:34:12.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.456+0000 7f09a9116640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09a4148c50 0x7f09a413b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:12.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.456+0000 7f09a9116640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09a4148c50 0x7f09a413b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45826/0 (socket says 172.21.15.5:45826) 2024-03-18T15:34:12.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.456+0000 7f09a9116640 1 -- 172.21.15.5:0/2461725488 learned_addr learned my addr 172.21.15.5:0/2461725488 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:12.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.456+0000 7f09a8915640 1 --2- 172.21.15.5:0/2461725488 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f09a40b4580 0x7f09a413e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:12.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.456+0000 7f09a3fff640 1 --2- 172.21.15.5:0/2461725488 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09a40b6ad0 0x7f09a413b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:12.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.456+0000 7f09a9116640 1 -- 172.21.15.5:0/2461725488 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f09a40b4580 msgr2=0x7f09a413e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:12.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.456+0000 7f09a9116640 1 --2- 172.21.15.5:0/2461725488 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f09a40b4580 0x7f09a413e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:12.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.456+0000 7f09a9116640 1 -- 172.21.15.5:0/2461725488 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09a40b6ad0 msgr2=0x7f09a413b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:12.460 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.456+0000 7f09a9116640 1 --2- 172.21.15.5:0/2461725488 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09a40b6ad0 0x7f09a413b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:12.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.456+0000 7f09a9116640 1 -- 172.21.15.5:0/2461725488 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09a413d000 con 0x7f09a4148c50 2024-03-18T15:34:12.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.456+0000 7f09a8915640 1 --2- 172.21.15.5:0/2461725488 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f09a40b4580 0x7f09a413e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:34:12.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.456+0000 7f09a9116640 1 --2- 172.21.15.5:0/2461725488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09a4148c50 0x7f09a413b710 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f099400c780 tx=0x7f099400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:12.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.456+0000 7f09a17fa640 1 -- 172.21.15.5:0/2461725488 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0994016020 con 0x7f09a4148c50 2024-03-18T15:34:12.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.456+0000 7f09a17fa640 1 -- 172.21.15.5:0/2461725488 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f099400a840 con 0x7f09a4148c50 2024-03-18T15:34:12.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.456+0000 7f09aab79640 1 -- 172.21.15.5:0/2461725488 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09a413dd80 con 0x7f09a4148c50 2024-03-18T15:34:12.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.456+0000 7f09a17fa640 1 -- 172.21.15.5:0/2461725488 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099400ac10 con 0x7f09a4148c50 2024-03-18T15:34:12.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.456+0000 7f09aab79640 1 -- 172.21.15.5:0/2461725488 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09a40b1710 con 0x7f09a4148c50 2024-03-18T15:34:12.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.456+0000 7f09aab79640 1 -- 172.21.15.5:0/2461725488 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09a404f550 con 0x7f09a4148c50 2024-03-18T15:34:12.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.456+0000 7f09a17fa640 1 -- 172.21.15.5:0/2461725488 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f0994019020 con 0x7f09a4148c50 2024-03-18T15:34:12.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.456+0000 7f09a17fa640 1 --2- 172.21.15.5:0/2461725488 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0990067660 0x7f0990069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:12.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.456+0000 7f09a17fa640 1 -- 172.21.15.5:0/2461725488 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0994085920 con 0x7f09a4148c50 2024-03-18T15:34:12.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.456+0000 7f09a8915640 1 --2- 172.21.15.5:0/2461725488 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0990067660 0x7f0990069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:12.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.456+0000 7f09a8915640 1 --2- 172.21.15.5:0/2461725488 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0990067660 0x7f0990069b20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f099c0af000 tx=0x7f099c0a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:12.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.460+0000 7f09a17fa640 1 -- 172.21.15.5:0/2461725488 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f0994020020 con 0x7f09a4148c50 2024-03-18T15:34:12.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.652+0000 7f09aab79640 1 -- 172.21.15.5:0/2461725488 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f09a4142fc0 con 0x7f09a4148c50 2024-03-18T15:34:12.655 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-03-18T15:34:12.652+0000 7f3da0624640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-18T15:34:12.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.652+0000 7f09a17fa640 1 -- 172.21.15.5:0/2461725488 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7f09940102e0 con 0x7f09a4148c50 2024-03-18T15:34:12.657 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-18T15:34:12.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.656+0000 7f09aab79640 1 -- 172.21.15.5:0/2461725488 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0990067660 msgr2=0x7f0990069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:12.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.656+0000 7f09aab79640 1 --2- 172.21.15.5:0/2461725488 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0990067660 0x7f0990069b20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f099c0af000 tx=0x7f099c0a7000 comp rx=0 tx=0).stop 2024-03-18T15:34:12.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.656+0000 7f09aab79640 1 -- 172.21.15.5:0/2461725488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09a4148c50 msgr2=0x7f09a413b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:12.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.656+0000 7f09aab79640 1 --2- 172.21.15.5:0/2461725488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09a4148c50 0x7f09a413b710 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f099400c780 tx=0x7f099400cc40 comp rx=0 tx=0).stop 2024-03-18T15:34:12.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.656+0000 7f09aab79640 1 -- 172.21.15.5:0/2461725488 shutdown_connections 2024-03-18T15:34:12.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.656+0000 7f09aab79640 1 --2- 172.21.15.5:0/2461725488 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0990067660 0x7f0990069b20 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:12.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.656+0000 7f09aab79640 1 --2- 172.21.15.5:0/2461725488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f09a4148c50 0x7f09a413b710 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:12.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.656+0000 7f09aab79640 1 --2- 172.21.15.5:0/2461725488 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09a40b6ad0 0x7f09a413b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:12.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.656+0000 7f09aab79640 1 --2- 172.21.15.5:0/2461725488 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f09a40b4580 0x7f09a413e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:12.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.656+0000 7f09aab79640 1 -- 172.21.15.5:0/2461725488 >> 172.21.15.5:0/2461725488 conn(0x7f09a40bacb0 msgr2=0x7f09a40b9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:12.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.656+0000 7f09aab79640 1 -- 172.21.15.5:0/2461725488 shutdown_connections 2024-03-18T15:34:12.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.656+0000 7f09aab79640 1 -- 172.21.15.5:0/2461725488 wait complete. 2024-03-18T15:34:12.663 INFO:tasks.ceph.mds.b.smithi080.stderr: -1100> 2024-03-18T15:33:48.008+0000 7fddb8a20640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-18T15:34:12.676 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-18T15:34:12.700 INFO:tasks.ceph.mds.b.smithi080.stdout:starting mds.b at 2024-03-18T15:34:12.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.784+0000 7f536f22c640 1 Processor -- start 2024-03-18T15:34:12.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.784+0000 7f536f22c640 1 -- start start 2024-03-18T15:34:12.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.788+0000 7f536f22c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5368072b40 0x7f536806d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:12.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.788+0000 7f536f22c640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53680afb70 con 0x7f53680b26a0 2024-03-18T15:34:12.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.788+0000 7f536f22c640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53680b4a20 con 0x7f53680af790 2024-03-18T15:34:12.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.788+0000 7f536f22c640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f536806d5b0 con 0x7f5368072b40 2024-03-18T15:34:12.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.788+0000 7f536d7c9640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f53680b26a0 0x7f53680b2af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51528/0 (socket says 172.21.15.5:51528) 2024-03-18T15:34:12.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.788+0000 7f536d7c9640 1 -- 172.21.15.5:0/1481720384 learned_addr learned my addr 172.21.15.5:0/1481720384 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:12.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.788+0000 7f5367fff640 1 --2- 172.21.15.5:0/1481720384 >> v2:172.21.15.80:3300/0 conn(0x7f5368072b40 0x7f536806d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:12.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.788+0000 7f53677fe640 1 -- 172.21.15.5:0/1481720384 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 797022771 0 0) 0x7f53680afb70 con 0x7f53680b26a0 2024-03-18T15:34:12.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.788+0000 7f53677fe640 1 -- 172.21.15.5:0/1481720384 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5354003530 con 0x7f53680b26a0 2024-03-18T15:34:12.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.788+0000 7f53677fe640 1 -- 172.21.15.5:0/1481720384 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1421801223 0 0) 0x7f5354003530 con 0x7f53680b26a0 2024-03-18T15:34:12.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.788+0000 7f53677fe640 1 -- 172.21.15.5:0/1481720384 >> v2:172.21.15.80:3300/0 conn(0x7f5368072b40 msgr2=0x7f536806d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:12.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.788+0000 7f53677fe640 1 --2- 172.21.15.5:0/1481720384 >> v2:172.21.15.80:3300/0 conn(0x7f5368072b40 0x7f536806d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:12.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.788+0000 7f53677fe640 1 -- 172.21.15.5:0/1481720384 >> v1:172.21.15.80:6789/0 conn(0x7f53680af790 legacy=0x7f5368147780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:12.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.788+0000 7f53677fe640 1 -- 172.21.15.5:0/1481720384 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53680b4d00 con 0x7f53680b26a0 2024-03-18T15:34:12.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.788+0000 7f53677fe640 1 -- 172.21.15.5:0/1481720384 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3837549003 0 0) 0x7f5360001680 con 0x7f53680b26a0 2024-03-18T15:34:12.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.788+0000 7f53677fe640 1 -- 172.21.15.5:0/1481720384 >> v1:172.21.15.5:6789/0 conn(0x7f53680b26a0 legacy=0x7f53680b2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:12.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.788+0000 7f53677fe640 1 --2- 172.21.15.5:0/1481720384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5354003aa0 0x7f5354003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:12.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.788+0000 7f53677fe640 1 --2- 172.21.15.5:0/1481720384 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5354004810 0x7f5354004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:12.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.788+0000 7f53677fe640 1 --2- 172.21.15.5:0/1481720384 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f53540052a0 0x7f535400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:12.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.788+0000 7f53677fe640 1 -- 172.21.15.5:0/1481720384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f536806d5b0 con 0x7f5354003aa0 2024-03-18T15:34:12.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.788+0000 7f53677fe640 1 -- 172.21.15.5:0/1481720384 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f535400c1f0 con 0x7f53540052a0 2024-03-18T15:34:12.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.788+0000 7f53677fe640 1 -- 172.21.15.5:0/1481720384 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f535400c3f0 con 0x7f5354004810 2024-03-18T15:34:12.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.788+0000 7f536cfc8640 1 --2- 172.21.15.5:0/1481720384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5354003aa0 0x7f5354003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:12.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.788+0000 7f5367fff640 1 --2- 172.21.15.5:0/1481720384 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5354004810 0x7f5354004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:12.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.788+0000 7f536d7c9640 1 --2- 172.21.15.5:0/1481720384 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f53540052a0 0x7f535400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:12.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.788+0000 7f536cfc8640 1 -- 172.21.15.5:0/1481720384 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5354004810 msgr2=0x7f5354004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:12.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.788+0000 7f536cfc8640 1 --2- 172.21.15.5:0/1481720384 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5354004810 0x7f5354004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:12.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.788+0000 7f536cfc8640 1 -- 172.21.15.5:0/1481720384 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f53540052a0 msgr2=0x7f535400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:12.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.788+0000 7f536cfc8640 1 --2- 172.21.15.5:0/1481720384 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f53540052a0 0x7f535400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:12.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.788+0000 7f536cfc8640 1 -- 172.21.15.5:0/1481720384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5354003530 con 0x7f5354003aa0 2024-03-18T15:34:12.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.788+0000 7f5367fff640 1 --2- 172.21.15.5:0/1481720384 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5354004810 0x7f5354004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:12.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.788+0000 7f536d7c9640 1 --2- 172.21.15.5:0/1481720384 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f53540052a0 0x7f535400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:12.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.788+0000 7f536d7c9640 1 -- 172.21.15.5:0/1481720384 reap_dead start 2024-03-18T15:34:12.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.792+0000 7f536cfc8640 1 --2- 172.21.15.5:0/1481720384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5354003aa0 0x7f5354003ec0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5358001780 tx=0x7f535801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=989b19a7faf8caa4 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:12.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.792+0000 7f53677fe640 1 -- 172.21.15.5:0/1481720384 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53580013a0 con 0x7f5354003aa0 2024-03-18T15:34:12.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.792+0000 7f53677fe640 1 -- 172.21.15.5:0/1481720384 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53580052f0 con 0x7f5354003aa0 2024-03-18T15:34:12.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.792+0000 7f53677fe640 1 -- 172.21.15.5:0/1481720384 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5358003840 con 0x7f5354003aa0 2024-03-18T15:34:12.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.792+0000 7f536f22c640 1 -- 172.21.15.5:0/1481720384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5354003aa0 msgr2=0x7f5354003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:12.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.792+0000 7f536f22c640 1 --2- 172.21.15.5:0/1481720384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5354003aa0 0x7f5354003ec0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5358001780 tx=0x7f535801c400 comp rx=0 tx=0).stop 2024-03-18T15:34:12.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.792+0000 7f536f22c640 1 -- 172.21.15.5:0/1481720384 shutdown_connections 2024-03-18T15:34:12.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.792+0000 7f536f22c640 1 --2- 172.21.15.5:0/1481720384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5354003aa0 0x7f5354003ec0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:12.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.792+0000 7f536f22c640 1 -- 172.21.15.5:0/1481720384 >> 172.21.15.5:0/1481720384 conn(0x7f53680b5670 msgr2=0x7f53680b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:12.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.792+0000 7f536f22c640 1 -- 172.21.15.5:0/1481720384 shutdown_connections 2024-03-18T15:34:12.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.792+0000 7f536f22c640 1 -- 172.21.15.5:0/1481720384 wait complete. 2024-03-18T15:34:12.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.792+0000 7f536f22c640 1 Processor -- start 2024-03-18T15:34:12.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.792+0000 7f536f22c640 1 -- start start 2024-03-18T15:34:12.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.792+0000 7f536f22c640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5354003aa0 0x7f5368137c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:12.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.792+0000 7f536f22c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53681e17e0 0x7f5368146330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:12.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.792+0000 7f536f22c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f536813de80 0x7f536813e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:12.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.792+0000 7f536f22c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5368147990 con 0x7f53681e17e0 2024-03-18T15:34:12.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.792+0000 7f536f22c640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f5368072f20 con 0x7f536813de80 2024-03-18T15:34:12.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.792+0000 7f536f22c640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f5368147b10 con 0x7f5354003aa0 2024-03-18T15:34:12.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.792+0000 7f5367fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53681e17e0 0x7f5368146330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:12.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.792+0000 7f5367fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53681e17e0 0x7f5368146330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45850/0 (socket says 172.21.15.5:45850) 2024-03-18T15:34:12.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.792+0000 7f5367fff640 1 -- 172.21.15.5:0/1334954060 learned_addr learned my addr 172.21.15.5:0/1334954060 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:12.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.792+0000 7f536d7c9640 1 --2- 172.21.15.5:0/1334954060 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f536813de80 0x7f536813e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:12.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.792+0000 7f536cfc8640 1 --2- 172.21.15.5:0/1334954060 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5354003aa0 0x7f5368137c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:12.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.792+0000 7f5367fff640 1 -- 172.21.15.5:0/1334954060 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5354003aa0 msgr2=0x7f5368137c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:12.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.792+0000 7f5367fff640 1 --2- 172.21.15.5:0/1334954060 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5354003aa0 0x7f5368137c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:12.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.792+0000 7f5367fff640 1 -- 172.21.15.5:0/1334954060 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f536813de80 msgr2=0x7f536813e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:12.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.792+0000 7f5367fff640 1 --2- 172.21.15.5:0/1334954060 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f536813de80 0x7f536813e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:12.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.792+0000 7f5367fff640 1 -- 172.21.15.5:0/1334954060 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53681471b0 con 0x7f53681e17e0 2024-03-18T15:34:12.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.792+0000 7f536d7c9640 1 --2- 172.21.15.5:0/1334954060 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f536813de80 0x7f536813e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:12.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.792+0000 7f5367fff640 1 --2- 172.21.15.5:0/1334954060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53681e17e0 0x7f5368146330 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f535000f3d0 tx=0x7f535000f890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:12.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.792+0000 7f53657fa640 1 -- 172.21.15.5:0/1334954060 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53500039a0 con 0x7f53681e17e0 2024-03-18T15:34:12.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.792+0000 7f53657fa640 1 -- 172.21.15.5:0/1334954060 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5350003b40 con 0x7f53681e17e0 2024-03-18T15:34:12.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.792+0000 7f536f22c640 1 -- 172.21.15.5:0/1334954060 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53680b83b0 con 0x7f53681e17e0 2024-03-18T15:34:12.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.792+0000 7f53657fa640 1 -- 172.21.15.5:0/1334954060 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5350004020 con 0x7f53681e17e0 2024-03-18T15:34:12.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.792+0000 7f536f22c640 1 -- 172.21.15.5:0/1334954060 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5368145870 con 0x7f53681e17e0 2024-03-18T15:34:12.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.796+0000 7f536f22c640 1 -- 172.21.15.5:0/1334954060 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f536804f550 con 0x7f53681e17e0 2024-03-18T15:34:12.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.796+0000 7f53657fa640 1 -- 172.21.15.5:0/1334954060 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f535001d020 con 0x7f53681e17e0 2024-03-18T15:34:12.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.796+0000 7f53657fa640 1 --2- 172.21.15.5:0/1334954060 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5348067660 0x7f5348069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:12.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.796+0000 7f536cfc8640 1 --2- 172.21.15.5:0/1334954060 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5348067660 0x7f5348069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:12.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.796+0000 7f53657fa640 1 -- 172.21.15.5:0/1334954060 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5795+0+0 (secure 0 0 0) 0x7f5350086320 con 0x7f53681e17e0 2024-03-18T15:34:12.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.796+0000 7f536cfc8640 1 --2- 172.21.15.5:0/1334954060 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5348067660 0x7f5348069b20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5358001750 tx=0x7f53580011a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:12.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.800+0000 7f53657fa640 1 -- 172.21.15.5:0/1334954060 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f5350024020 con 0x7f53681e17e0 2024-03-18T15:34:12.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:12.992+0000 7f536f22c640 1 -- 172.21.15.5:0/1334954060 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f53680b8580 con 0x7f53681e17e0 2024-03-18T15:34:13.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.656+0000 7f53657fa640 1 -- 172.21.15.5:0/1334954060 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7f5350056f30 con 0x7f53681e17e0 2024-03-18T15:34:13.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.660+0000 7f536f22c640 1 -- 172.21.15.5:0/1334954060 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5348067660 msgr2=0x7f5348069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:13.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.660+0000 7f536f22c640 1 --2- 172.21.15.5:0/1334954060 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5348067660 0x7f5348069b20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5358001750 tx=0x7f53580011a0 comp rx=0 tx=0).stop 2024-03-18T15:34:13.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.660+0000 7f536f22c640 1 -- 172.21.15.5:0/1334954060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53681e17e0 msgr2=0x7f5368146330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:13.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.660+0000 7f536f22c640 1 --2- 172.21.15.5:0/1334954060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53681e17e0 0x7f5368146330 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f535000f3d0 tx=0x7f535000f890 comp rx=0 tx=0).stop 2024-03-18T15:34:13.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.660+0000 7f536f22c640 1 -- 172.21.15.5:0/1334954060 shutdown_connections 2024-03-18T15:34:13.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.660+0000 7f536f22c640 1 --2- 172.21.15.5:0/1334954060 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5348067660 0x7f5348069b20 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:13.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.660+0000 7f536f22c640 1 --2- 172.21.15.5:0/1334954060 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f536813de80 0x7f536813e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:13.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.660+0000 7f536f22c640 1 --2- 172.21.15.5:0/1334954060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53681e17e0 0x7f5368146330 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:13.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.660+0000 7f536f22c640 1 --2- 172.21.15.5:0/1334954060 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5354003aa0 0x7f5368137c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:13.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.660+0000 7f536f22c640 1 -- 172.21.15.5:0/1334954060 >> 172.21.15.5:0/1334954060 conn(0x7f53680b5670 msgr2=0x7f53680ae9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:13.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.660+0000 7f536f22c640 1 -- 172.21.15.5:0/1334954060 shutdown_connections 2024-03-18T15:34:13.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.660+0000 7f536f22c640 1 -- 172.21.15.5:0/1334954060 wait complete. 2024-03-18T15:34:13.684 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-18T15:34:13.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.812+0000 7f3dfc71d640 1 Processor -- start 2024-03-18T15:34:13.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.812+0000 7f3dfc71d640 1 -- start start 2024-03-18T15:34:13.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.812+0000 7f3dfc71d640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3df406d300 0x7f3df40aea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:13.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.812+0000 7f3dfc71d640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3df406d6e0 con 0x7f3df40ad8e0 2024-03-18T15:34:13.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.812+0000 7f3dfc71d640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3df40b4d20 con 0x7f3df414b230 2024-03-18T15:34:13.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.812+0000 7f3dfc71d640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f3df40b5370 con 0x7f3df406d300 2024-03-18T15:34:13.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.812+0000 7f3df9cb8640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f3df40ad8e0 0x7f3df414ab20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51540/0 (socket says 172.21.15.5:51540) 2024-03-18T15:34:13.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.812+0000 7f3df9cb8640 1 -- 172.21.15.5:0/2343682783 learned_addr learned my addr 172.21.15.5:0/2343682783 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:13.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.816+0000 7f3dfa4b9640 1 --2- 172.21.15.5:0/2343682783 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3df406d300 0x7f3df40aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:13.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.816+0000 7f3df94b7640 1 -- 172.21.15.5:0/2343682783 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3672619283 0 0) 0x7f3df406d6e0 con 0x7f3df40ad8e0 2024-03-18T15:34:13.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.816+0000 7f3df94b7640 1 -- 172.21.15.5:0/2343682783 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3de0003530 con 0x7f3df40ad8e0 2024-03-18T15:34:13.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.816+0000 7f3df94b7640 1 -- 172.21.15.5:0/2343682783 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1607206886 0 0) 0x7f3de0003530 con 0x7f3df40ad8e0 2024-03-18T15:34:13.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.816+0000 7f3df94b7640 1 -- 172.21.15.5:0/2343682783 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3df406d300 msgr2=0x7f3df40aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:13.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.816+0000 7f3df94b7640 1 --2- 172.21.15.5:0/2343682783 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3df406d300 0x7f3df40aea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:13.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.816+0000 7f3df94b7640 1 -- 172.21.15.5:0/2343682783 >> v1:172.21.15.80:6789/0 conn(0x7f3df414b230 legacy=0x7f3df4072490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:13.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.816+0000 7f3df94b7640 1 -- 172.21.15.5:0/2343682783 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3df40b5000 con 0x7f3df40ad8e0 2024-03-18T15:34:13.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.816+0000 7f3df94b7640 1 -- 172.21.15.5:0/2343682783 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3837549003 0 0) 0x7f3df0001a00 con 0x7f3df40ad8e0 2024-03-18T15:34:13.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.816+0000 7f3df94b7640 1 -- 172.21.15.5:0/2343682783 >> v1:172.21.15.5:6789/0 conn(0x7f3df40ad8e0 legacy=0x7f3df414ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:13.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.816+0000 7f3df94b7640 1 --2- 172.21.15.5:0/2343682783 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3df406d300 0x7f3de0003ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:13.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.816+0000 7f3df94b7640 1 --2- 172.21.15.5:0/2343682783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3de0004050 0x7f3de0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:13.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.816+0000 7f3df94b7640 1 --2- 172.21.15.5:0/2343682783 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3de0004d50 0x7f3de00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:13.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.816+0000 7f3df94b7640 1 -- 172.21.15.5:0/2343682783 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3de000bc00 con 0x7f3de0004050 2024-03-18T15:34:13.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.816+0000 7f3df94b7640 1 -- 172.21.15.5:0/2343682783 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3de000bdb0 con 0x7f3de0004d50 2024-03-18T15:34:13.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.816+0000 7f3df94b7640 1 -- 172.21.15.5:0/2343682783 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f3de000bfb0 con 0x7f3df406d300 2024-03-18T15:34:13.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.816+0000 7f3df9cb8640 1 --2- 172.21.15.5:0/2343682783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3de0004050 0x7f3de0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:13.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.816+0000 7f3dfacba640 1 --2- 172.21.15.5:0/2343682783 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3de0004d50 0x7f3de00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:13.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.816+0000 7f3dfa4b9640 1 --2- 172.21.15.5:0/2343682783 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3df406d300 0x7f3de0003ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:13.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.816+0000 7f3df9cb8640 1 -- 172.21.15.5:0/2343682783 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3df406d300 msgr2=0x7f3de0003ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:13.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.816+0000 7f3df9cb8640 1 --2- 172.21.15.5:0/2343682783 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3df406d300 0x7f3de0003ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:13.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.816+0000 7f3df9cb8640 1 -- 172.21.15.5:0/2343682783 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3de0004d50 msgr2=0x7f3de00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:13.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.816+0000 7f3df9cb8640 1 --2- 172.21.15.5:0/2343682783 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3de0004d50 0x7f3de00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:13.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.816+0000 7f3df9cb8640 1 -- 172.21.15.5:0/2343682783 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3de0003530 con 0x7f3de0004050 2024-03-18T15:34:13.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.816+0000 7f3dfa4b9640 1 --2- 172.21.15.5:0/2343682783 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3df406d300 0x7f3de0003ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:13.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.816+0000 7f3dfacba640 1 --2- 172.21.15.5:0/2343682783 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3de0004d50 0x7f3de00051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:13.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.816+0000 7f3df9cb8640 1 --2- 172.21.15.5:0/2343682783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3de0004050 0x7f3de0004810 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3df000a840 tx=0x7f3df001d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=205166ef7f34f42e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:13.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.816+0000 7f3df94b7640 1 -- 172.21.15.5:0/2343682783 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df000a460 con 0x7f3de0004050 2024-03-18T15:34:13.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.816+0000 7f3df94b7640 1 -- 172.21.15.5:0/2343682783 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3df00ad650 con 0x7f3de0004050 2024-03-18T15:34:13.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.816+0000 7f3df94b7640 1 -- 172.21.15.5:0/2343682783 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df00ada80 con 0x7f3de0004050 2024-03-18T15:34:13.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.816+0000 7f3dfc71d640 1 -- 172.21.15.5:0/2343682783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3de0004050 msgr2=0x7f3de0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:13.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.816+0000 7f3dfc71d640 1 --2- 172.21.15.5:0/2343682783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3de0004050 0x7f3de0004810 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3df000a840 tx=0x7f3df001d360 comp rx=0 tx=0).stop 2024-03-18T15:34:13.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.816+0000 7f3dfacba640 1 -- 172.21.15.5:0/2343682783 reap_dead start 2024-03-18T15:34:13.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.816+0000 7f3dfc71d640 1 -- 172.21.15.5:0/2343682783 shutdown_connections 2024-03-18T15:34:13.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.816+0000 7f3dfc71d640 1 -- 172.21.15.5:0/2343682783 >> 172.21.15.5:0/2343682783 conn(0x7f3df40b5670 msgr2=0x7f3df40b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:13.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.816+0000 7f3dfc71d640 1 -- 172.21.15.5:0/2343682783 shutdown_connections 2024-03-18T15:34:13.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.816+0000 7f3dfc71d640 1 -- 172.21.15.5:0/2343682783 wait complete. 2024-03-18T15:34:13.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.820+0000 7f3dfc71d640 1 Processor -- start 2024-03-18T15:34:13.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.820+0000 7f3dfc71d640 1 -- start start 2024-03-18T15:34:13.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.820+0000 7f3dfc71d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3de0004050 0x7f3df414a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:13.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.820+0000 7f3dfc71d640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3df40b1460 0x7f3df41e0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:13.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.820+0000 7f3dfc71d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3df4142220 0x7f3df413cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:13.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.820+0000 7f3dfc71d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3df41538f0 con 0x7f3df4142220 2024-03-18T15:34:13.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.820+0000 7f3dfc71d640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3df4153770 con 0x7f3de0004050 2024-03-18T15:34:13.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.820+0000 7f3dfc71d640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f3df4153a70 con 0x7f3df40b1460 2024-03-18T15:34:13.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.820+0000 7f3dfacba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3df4142220 0x7f3df413cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:13.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.820+0000 7f3dfacba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3df4142220 0x7f3df413cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45870/0 (socket says 172.21.15.5:45870) 2024-03-18T15:34:13.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.820+0000 7f3dfacba640 1 -- 172.21.15.5:0/2881283332 learned_addr learned my addr 172.21.15.5:0/2881283332 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:13.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.820+0000 7f3dfa4b9640 1 --2- 172.21.15.5:0/2881283332 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3de0004050 0x7f3df414a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:13.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.820+0000 7f3df9cb8640 1 --2- 172.21.15.5:0/2881283332 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3df40b1460 0x7f3df41e0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:13.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.820+0000 7f3dfacba640 1 -- 172.21.15.5:0/2881283332 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3df40b1460 msgr2=0x7f3df41e0350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:13.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.820+0000 7f3dfacba640 1 --2- 172.21.15.5:0/2881283332 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3df40b1460 0x7f3df41e0350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:13.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.820+0000 7f3dfacba640 1 -- 172.21.15.5:0/2881283332 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3de0004050 msgr2=0x7f3df414a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:13.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.820+0000 7f3dfacba640 1 --2- 172.21.15.5:0/2881283332 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3de0004050 0x7f3df414a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:13.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.820+0000 7f3dfacba640 1 -- 172.21.15.5:0/2881283332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3df4143790 con 0x7f3df4142220 2024-03-18T15:34:13.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.820+0000 7f3df9cb8640 1 --2- 172.21.15.5:0/2881283332 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3df40b1460 0x7f3df41e0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:13.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.820+0000 7f3dfacba640 1 --2- 172.21.15.5:0/2881283332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3df4142220 0x7f3df413cb40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3df414b610 tx=0x7f3dec00d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:13.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.820+0000 7f3de6ffd640 1 -- 172.21.15.5:0/2881283332 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dec007030 con 0x7f3df4142220 2024-03-18T15:34:13.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.820+0000 7f3de6ffd640 1 -- 172.21.15.5:0/2881283332 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dec0071f0 con 0x7f3df4142220 2024-03-18T15:34:13.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.820+0000 7f3de6ffd640 1 -- 172.21.15.5:0/2881283332 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dec0075e0 con 0x7f3df4142220 2024-03-18T15:34:13.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.820+0000 7f3dfc71d640 1 -- 172.21.15.5:0/2881283332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3df41426d0 con 0x7f3df4142220 2024-03-18T15:34:13.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.820+0000 7f3dfc71d640 1 -- 172.21.15.5:0/2881283332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3df41498c0 con 0x7f3df4142220 2024-03-18T15:34:13.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.820+0000 7f3dfc71d640 1 -- 172.21.15.5:0/2881283332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3df404f5c0 con 0x7f3df4142220 2024-03-18T15:34:13.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.824+0000 7f3de6ffd640 1 -- 172.21.15.5:0/2881283332 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f3dec01b020 con 0x7f3df4142220 2024-03-18T15:34:13.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.824+0000 7f3de6ffd640 1 --2- 172.21.15.5:0/2881283332 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3dd8067660 0x7f3dd8069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:13.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.824+0000 7f3de6ffd640 1 -- 172.21.15.5:0/2881283332 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5795+0+0 (secure 0 0 0) 0x7f3dec085190 con 0x7f3df4142220 2024-03-18T15:34:13.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.824+0000 7f3dfa4b9640 1 --2- 172.21.15.5:0/2881283332 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3dd8067660 0x7f3dd8069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:13.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.824+0000 7f3dfa4b9640 1 --2- 172.21.15.5:0/2881283332 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3dd8067660 0x7f3dd8069b20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3de8009ac0 tx=0x7f3de8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:13.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.828+0000 7f3de6ffd640 1 -- 172.21.15.5:0/2881283332 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f3dec055e40 con 0x7f3df4142220 2024-03-18T15:34:13.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:13.992+0000 7f3dfc71d640 1 -- 172.21.15.5:0/2881283332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3df4145e20 con 0x7f3df4142220 2024-03-18T15:34:14.593 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-18T15:34:14.596 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-18T15:34:14.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.664+0000 7f3de6ffd640 1 -- 172.21.15.5:0/2881283332 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v23) ==== 181+0+0 (secure 0 0 0) 0x7f3dec055740 con 0x7f3df4142220 2024-03-18T15:34:14.669 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' removed 2024-03-18T15:34:14.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.668+0000 7f3dfc71d640 1 -- 172.21.15.5:0/2881283332 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3dd8067660 msgr2=0x7f3dd8069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:14.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.668+0000 7f3dfc71d640 1 --2- 172.21.15.5:0/2881283332 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3dd8067660 0x7f3dd8069b20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3de8009ac0 tx=0x7f3de8008040 comp rx=0 tx=0).stop 2024-03-18T15:34:14.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.668+0000 7f3dfc71d640 1 -- 172.21.15.5:0/2881283332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3df4142220 msgr2=0x7f3df413cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:14.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.668+0000 7f3dfc71d640 1 --2- 172.21.15.5:0/2881283332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3df4142220 0x7f3df413cb40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3df414b610 tx=0x7f3dec00d890 comp rx=0 tx=0).stop 2024-03-18T15:34:14.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.668+0000 7f3dfc71d640 1 -- 172.21.15.5:0/2881283332 shutdown_connections 2024-03-18T15:34:14.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.668+0000 7f3dfc71d640 1 --2- 172.21.15.5:0/2881283332 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3dd8067660 0x7f3dd8069b20 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:14.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.668+0000 7f3dfc71d640 1 --2- 172.21.15.5:0/2881283332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3df4142220 0x7f3df413cb40 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:14.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.668+0000 7f3dfc71d640 1 --2- 172.21.15.5:0/2881283332 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3df40b1460 0x7f3df41e0350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:14.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.668+0000 7f3dfc71d640 1 --2- 172.21.15.5:0/2881283332 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3de0004050 0x7f3df414a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:14.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.668+0000 7f3dfc71d640 1 -- 172.21.15.5:0/2881283332 >> 172.21.15.5:0/2881283332 conn(0x7f3df40b5670 msgr2=0x7f3df4147dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:14.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.668+0000 7f3dfc71d640 1 -- 172.21.15.5:0/2881283332 shutdown_connections 2024-03-18T15:34:14.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.668+0000 7f3dfc71d640 1 -- 172.21.15.5:0/2881283332 wait complete. 2024-03-18T15:34:14.695 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-18T15:34:14.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.864+0000 7fa8d0b8e640 1 Processor -- start 2024-03-18T15:34:14.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.864+0000 7fa8d0b8e640 1 -- start start 2024-03-18T15:34:14.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.868+0000 7fa8d0b8e640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa8cc06d300 0x7fa8cc145250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:14.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.868+0000 7fa8d0b8e640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa8cc06f640 0x7fa8cc14fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:14.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.868+0000 7fa8d0b8e640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8cc0afd60 con 0x7fa8cc0b4820 2024-03-18T15:34:14.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.868+0000 7fa8d0b8e640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fa8cc0b52b0 con 0x7fa8cc06d300 2024-03-18T15:34:14.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.868+0000 7fa8d0b8e640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fa8cc06d6e0 con 0x7fa8cc06f640 2024-03-18T15:34:14.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.868+0000 7fa8cad9d640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa8cc06f640 0x7fa8cc14fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:14.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.868+0000 7fa8ca59c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa8cc06d300 0x7fa8cc145250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:14.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.868+0000 7fa8cad9d640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa8cc06f640 0x7fa8cc14fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:54848/0 (socket says 172.21.15.5:54848) 2024-03-18T15:34:14.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.868+0000 7fa8ca59c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa8cc06d300 0x7fa8cc145250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:48984/0 (socket says 172.21.15.5:48984) 2024-03-18T15:34:14.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.868+0000 7fa8cad9d640 1 -- 172.21.15.5:0/2919235561 learned_addr learned my addr 172.21.15.5:0/2919235561 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:14.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.868+0000 7fa8ca59c640 1 -- 172.21.15.5:0/2919235561 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa8cc06f640 msgr2=0x7fa8cc14fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:14.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.868+0000 7fa8ca59c640 1 --2- 172.21.15.5:0/2919235561 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa8cc06f640 0x7fa8cc14fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:14.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.868+0000 7fa8ca59c640 1 -- 172.21.15.5:0/2919235561 >> v1:172.21.15.80:6789/0 conn(0x7fa8cc0b4820 legacy=0x7fa8cc13efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:14.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.868+0000 7fa8ca59c640 1 -- 172.21.15.5:0/2919235561 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8cc070960 con 0x7fa8cc06d300 2024-03-18T15:34:14.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.868+0000 7fa8ca59c640 1 --2- 172.21.15.5:0/2919235561 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa8cc06d300 0x7fa8cc145250 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc0097c0 tx=0x7fa8bc01c350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4de1db7216961b70 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:14.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.868+0000 7fa8c959a640 1 -- 172.21.15.5:0/2919235561 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc0b0070 con 0x7fa8cc06d300 2024-03-18T15:34:14.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.868+0000 7fa8c959a640 1 -- 172.21.15.5:0/2919235561 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8bc004230 con 0x7fa8cc06d300 2024-03-18T15:34:14.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.868+0000 7fa8c959a640 1 -- 172.21.15.5:0/2919235561 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc003620 con 0x7fa8cc06d300 2024-03-18T15:34:14.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.872+0000 7fa8d0b8e640 1 -- 172.21.15.5:0/2919235561 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa8cc06d300 msgr2=0x7fa8cc145250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:14.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.872+0000 7fa8d0b8e640 1 --2- 172.21.15.5:0/2919235561 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa8cc06d300 0x7fa8cc145250 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc0097c0 tx=0x7fa8bc01c350 comp rx=0 tx=0).stop 2024-03-18T15:34:14.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.872+0000 7fa8d0b8e640 1 -- 172.21.15.5:0/2919235561 shutdown_connections 2024-03-18T15:34:14.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.872+0000 7fa8d0b8e640 1 --2- 172.21.15.5:0/2919235561 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa8cc06f640 0x7fa8cc14fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:14.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.872+0000 7fa8d0b8e640 1 --2- 172.21.15.5:0/2919235561 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa8cc06d300 0x7fa8cc145250 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:14.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.872+0000 7fa8d0b8e640 1 -- 172.21.15.5:0/2919235561 >> 172.21.15.5:0/2919235561 conn(0x7fa8cc0b5670 msgr2=0x7fa8cc0b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:14.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.872+0000 7fa8d0b8e640 1 -- 172.21.15.5:0/2919235561 shutdown_connections 2024-03-18T15:34:14.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.872+0000 7fa8d0b8e640 1 -- 172.21.15.5:0/2919235561 wait complete. 2024-03-18T15:34:14.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.872+0000 7fa8d0b8e640 1 Processor -- start 2024-03-18T15:34:14.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.872+0000 7fa8d0b8e640 1 -- start start 2024-03-18T15:34:14.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.872+0000 7fa8d0b8e640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa8cc06d300 0x7fa8cc13e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:14.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.872+0000 7fa8d0b8e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa8cc06f640 0x7fa8cc1dd990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:14.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.872+0000 7fa8d0b8e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8cc1dded0 0x7fa8cc138c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:14.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.872+0000 7fa8d0b8e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc0b2a70 con 0x7fa8cc1dded0 2024-03-18T15:34:14.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.872+0000 7fa8d0b8e640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa8cc0b28f0 con 0x7fa8cc06f640 2024-03-18T15:34:14.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.872+0000 7fa8d0b8e640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fa8cc152890 con 0x7fa8cc06d300 2024-03-18T15:34:14.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.872+0000 7fa8cad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8cc1dded0 0x7fa8cc138c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:14.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.872+0000 7fa8cad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8cc1dded0 0x7fa8cc138c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45878/0 (socket says 172.21.15.5:45878) 2024-03-18T15:34:14.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.872+0000 7fa8cad9d640 1 -- 172.21.15.5:0/641074854 learned_addr learned my addr 172.21.15.5:0/641074854 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:14.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.872+0000 7fa8ca59c640 1 --2- 172.21.15.5:0/641074854 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa8cc06d300 0x7fa8cc13e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:14.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.872+0000 7fa8c9d9b640 1 --2- 172.21.15.5:0/641074854 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa8cc06f640 0x7fa8cc1dd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:14.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.872+0000 7fa8cad9d640 1 -- 172.21.15.5:0/641074854 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa8cc06d300 msgr2=0x7fa8cc13e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:14.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.872+0000 7fa8cad9d640 1 --2- 172.21.15.5:0/641074854 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa8cc06d300 0x7fa8cc13e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:14.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.872+0000 7fa8cad9d640 1 -- 172.21.15.5:0/641074854 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa8cc06f640 msgr2=0x7fa8cc1dd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:14.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.872+0000 7fa8cad9d640 1 --2- 172.21.15.5:0/641074854 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa8cc06f640 0x7fa8cc1dd990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:14.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.872+0000 7fa8cad9d640 1 -- 172.21.15.5:0/641074854 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8cc141130 con 0x7fa8cc1dded0 2024-03-18T15:34:14.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.872+0000 7fa8cad9d640 1 --2- 172.21.15.5:0/641074854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8cc1dded0 0x7fa8cc138c20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa8c400efc0 tx=0x7fa8c400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:14.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.872+0000 7fa8baffd640 1 -- 172.21.15.5:0/641074854 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c400cd20 con 0x7fa8cc1dded0 2024-03-18T15:34:14.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.872+0000 7fa8baffd640 1 -- 172.21.15.5:0/641074854 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8c40039a0 con 0x7fa8cc1dded0 2024-03-18T15:34:14.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.872+0000 7fa8d0b8e640 1 -- 172.21.15.5:0/641074854 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8cc13ea20 con 0x7fa8cc1dded0 2024-03-18T15:34:14.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.872+0000 7fa8baffd640 1 -- 172.21.15.5:0/641074854 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8c4003d90 con 0x7fa8cc1dded0 2024-03-18T15:34:14.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.872+0000 7fa8d0b8e640 1 -- 172.21.15.5:0/641074854 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8cc140e50 con 0x7fa8cc1dded0 2024-03-18T15:34:14.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.876+0000 7fa8d0b8e640 1 -- 172.21.15.5:0/641074854 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8cc13fc20 con 0x7fa8cc1dded0 2024-03-18T15:34:14.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.876+0000 7fa8baffd640 1 -- 172.21.15.5:0/641074854 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fa8c4012050 con 0x7fa8cc1dded0 2024-03-18T15:34:14.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.876+0000 7fa8baffd640 1 --2- 172.21.15.5:0/641074854 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa8ac067660 0x7fa8ac069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:14.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.876+0000 7fa8ca59c640 1 --2- 172.21.15.5:0/641074854 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa8ac067660 0x7fa8ac069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:14.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.876+0000 7fa8baffd640 1 -- 172.21.15.5:0/641074854 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5357+0+0 (secure 0 0 0) 0x7fa8c4085be0 con 0x7fa8cc1dded0 2024-03-18T15:34:14.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.876+0000 7fa8ca59c640 1 --2- 172.21.15.5:0/641074854 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa8ac067660 0x7fa8ac069b20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc0ad000 tx=0x7fa8bc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:14.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:14.880+0000 7fa8baffd640 1 -- 172.21.15.5:0/641074854 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fa8c4010040 con 0x7fa8cc1dded0 2024-03-18T15:34:15.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.048+0000 7fa8d0b8e640 1 -- 172.21.15.5:0/641074854 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa8cc04f5c0 con 0x7fa8cc1dded0 2024-03-18T15:34:15.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.672+0000 7fa8baffd640 1 -- 172.21.15.5:0/641074854 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v24) ==== 169+0+0 (secure 0 0 0) 0x7fa8c4056a50 con 0x7fa8cc1dded0 2024-03-18T15:34:15.677 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' removed 2024-03-18T15:34:15.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.676+0000 7fa8d0b8e640 1 -- 172.21.15.5:0/641074854 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa8ac067660 msgr2=0x7fa8ac069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:15.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.676+0000 7fa8d0b8e640 1 --2- 172.21.15.5:0/641074854 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa8ac067660 0x7fa8ac069b20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc0ad000 tx=0x7fa8bc008040 comp rx=0 tx=0).stop 2024-03-18T15:34:15.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.676+0000 7fa8d0b8e640 1 -- 172.21.15.5:0/641074854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8cc1dded0 msgr2=0x7fa8cc138c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:15.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.676+0000 7fa8d0b8e640 1 --2- 172.21.15.5:0/641074854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8cc1dded0 0x7fa8cc138c20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa8c400efc0 tx=0x7fa8c400c490 comp rx=0 tx=0).stop 2024-03-18T15:34:15.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.676+0000 7fa8d0b8e640 1 -- 172.21.15.5:0/641074854 shutdown_connections 2024-03-18T15:34:15.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.676+0000 7fa8d0b8e640 1 --2- 172.21.15.5:0/641074854 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa8ac067660 0x7fa8ac069b20 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:15.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.676+0000 7fa8d0b8e640 1 --2- 172.21.15.5:0/641074854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8cc1dded0 0x7fa8cc138c20 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:15.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.676+0000 7fa8d0b8e640 1 --2- 172.21.15.5:0/641074854 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa8cc06f640 0x7fa8cc1dd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:15.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.676+0000 7fa8d0b8e640 1 --2- 172.21.15.5:0/641074854 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa8cc06d300 0x7fa8cc13e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:15.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.676+0000 7fa8d0b8e640 1 -- 172.21.15.5:0/641074854 >> 172.21.15.5:0/641074854 conn(0x7fa8cc0b5670 msgr2=0x7fa8cc0b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:15.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.680+0000 7fa8d0b8e640 1 -- 172.21.15.5:0/641074854 shutdown_connections 2024-03-18T15:34:15.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.680+0000 7fa8d0b8e640 1 -- 172.21.15.5:0/641074854 wait complete. 2024-03-18T15:34:15.706 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-18T15:34:15.706 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-18T15:34:15.706 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-18T15:34:15.707 INFO:teuthology.orchestra.run:waiting for 300 2024-03-18T15:34:15.707 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-18T15:34:15.707 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-18T15:34:15.707 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-18T15:34:15.707 INFO:teuthology.orchestra.run:waiting for 300 2024-03-18T15:34:15.717 INFO:tasks.ceph.mds.a:Stopped 2024-03-18T15:34:15.718 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-18T15:34:15.723 INFO:tasks.ceph.mds.b:Stopped 2024-03-18T15:34:15.723 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-18T15:34:15.767 INFO:tasks.ceph.mds.a:Started 2024-03-18T15:34:15.767 DEBUG:teuthology.parallel:result is None 2024-03-18T15:34:15.768 INFO:tasks.ceph.mds.b:Started 2024-03-18T15:34:15.768 DEBUG:teuthology.parallel:result is None 2024-03-18T15:34:15.768 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-18T15:34:15.839 INFO:tasks.ceph.mds.b.smithi080.stdout:starting mds.b at 2024-03-18T15:34:15.840 INFO:tasks.ceph.mds.a.smithi005.stdout:starting mds.a at 2024-03-18T15:34:15.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.892+0000 7fed08a3f640 1 Processor -- start 2024-03-18T15:34:15.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.892+0000 7fed08a3f640 1 -- start start 2024-03-18T15:34:15.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.892+0000 7fed08a3f640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fed040b5710 0x7fed0414a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:15.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.892+0000 7fed08a3f640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fed0414ae60 0x7fed04140ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:15.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.892+0000 7fed08a3f640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed040b9d50 con 0x7fed04141430 2024-03-18T15:34:15.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.892+0000 7fed08a3f640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fed040b4320 con 0x7fed0414ae60 2024-03-18T15:34:15.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.892+0000 7fed08a3f640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fed040b4570 con 0x7fed040b5710 2024-03-18T15:34:15.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.892+0000 7fed01d9b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fed0414ae60 0x7fed04140ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:15.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.892+0000 7fed01d9b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fed0414ae60 0x7fed04140ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45890/0 (socket says 172.21.15.5:45890) 2024-03-18T15:34:15.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.892+0000 7fed01d9b640 1 -- 172.21.15.5:0/2178048186 learned_addr learned my addr 172.21.15.5:0/2178048186 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:15.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.892+0000 7fed0259c640 1 --2- 172.21.15.5:0/2178048186 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fed040b5710 0x7fed0414a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:15.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.892+0000 7fed01d9b640 1 -- 172.21.15.5:0/2178048186 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fed040b5710 msgr2=0x7fed0414a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:15.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.892+0000 7fed01d9b640 1 --2- 172.21.15.5:0/2178048186 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fed040b5710 0x7fed0414a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:15.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.892+0000 7fed01d9b640 1 -- 172.21.15.5:0/2178048186 >> v1:172.21.15.80:6789/0 conn(0x7fed04141430 legacy=0x7fed040b7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:15.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.892+0000 7fed01d9b640 1 -- 172.21.15.5:0/2178048186 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed040b4c30 con 0x7fed0414ae60 2024-03-18T15:34:15.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.892+0000 7fed0259c640 1 --2- 172.21.15.5:0/2178048186 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fed040b5710 0x7fed0414a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:34:15.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.896+0000 7fed01d9b640 1 --2- 172.21.15.5:0/2178048186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed0414ae60 0x7fed04140ef0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fecf40098a0 tx=0x7fecf401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcbc79c100ccca2b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:15.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.896+0000 7fed0159a640 1 -- 172.21.15.5:0/2178048186 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf40b0070 con 0x7fed0414ae60 2024-03-18T15:34:15.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.896+0000 7fed0159a640 1 -- 172.21.15.5:0/2178048186 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecf40ab600 con 0x7fed0414ae60 2024-03-18T15:34:15.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.896+0000 7fed0159a640 1 -- 172.21.15.5:0/2178048186 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecf40ab9d0 con 0x7fed0414ae60 2024-03-18T15:34:15.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.896+0000 7fed08a3f640 1 -- 172.21.15.5:0/2178048186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed0414ae60 msgr2=0x7fed04140ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:15.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.896+0000 7fed08a3f640 1 --2- 172.21.15.5:0/2178048186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed0414ae60 0x7fed04140ef0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fecf40098a0 tx=0x7fecf401c420 comp rx=0 tx=0).stop 2024-03-18T15:34:15.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.896+0000 7fed08a3f640 1 -- 172.21.15.5:0/2178048186 shutdown_connections 2024-03-18T15:34:15.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.896+0000 7fed08a3f640 1 --2- 172.21.15.5:0/2178048186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed0414ae60 0x7fed04140ef0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:15.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.896+0000 7fed08a3f640 1 --2- 172.21.15.5:0/2178048186 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fed040b5710 0x7fed0414a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:15.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.896+0000 7fed08a3f640 1 -- 172.21.15.5:0/2178048186 >> 172.21.15.5:0/2178048186 conn(0x7fed040673c0 msgr2=0x7fed040677d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:15.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.896+0000 7fed08a3f640 1 -- 172.21.15.5:0/2178048186 shutdown_connections 2024-03-18T15:34:15.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.896+0000 7fed08a3f640 1 -- 172.21.15.5:0/2178048186 wait complete. 2024-03-18T15:34:15.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.896+0000 7fed08a3f640 1 Processor -- start 2024-03-18T15:34:15.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.896+0000 7fed08a3f640 1 -- start start 2024-03-18T15:34:15.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.896+0000 7fed08a3f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fed040b5710 0x7fed040bd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:15.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.896+0000 7fed08a3f640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fed04141430 0x7fed04139460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:15.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.896+0000 7fed08a3f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed0414ae60 0x7fed040bbad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:15.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.896+0000 7fed08a3f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fed04141840 con 0x7fed0414ae60 2024-03-18T15:34:15.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.896+0000 7fed08a3f640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fed0414b2c0 con 0x7fed040b5710 2024-03-18T15:34:15.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.896+0000 7fed08a3f640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fed041419c0 con 0x7fed04141430 2024-03-18T15:34:15.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.896+0000 7fed02d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed0414ae60 0x7fed040bbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:15.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.896+0000 7fed02d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed0414ae60 0x7fed040bbad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45892/0 (socket says 172.21.15.5:45892) 2024-03-18T15:34:15.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.896+0000 7fed02d9d640 1 -- 172.21.15.5:0/10217366 learned_addr learned my addr 172.21.15.5:0/10217366 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:15.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.896+0000 7fed0259c640 1 --2- 172.21.15.5:0/10217366 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fed040b5710 0x7fed040bd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:15.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.896+0000 7fed01d9b640 1 --2- 172.21.15.5:0/10217366 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fed04141430 0x7fed04139460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:15.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.896+0000 7fed02d9d640 1 -- 172.21.15.5:0/10217366 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fed04141430 msgr2=0x7fed04139460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:15.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.896+0000 7fed02d9d640 1 --2- 172.21.15.5:0/10217366 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fed04141430 0x7fed04139460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:15.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.896+0000 7fed02d9d640 1 -- 172.21.15.5:0/10217366 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fed040b5710 msgr2=0x7fed040bd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:15.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.896+0000 7fed02d9d640 1 --2- 172.21.15.5:0/10217366 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fed040b5710 0x7fed040bd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:15.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.896+0000 7fed02d9d640 1 -- 172.21.15.5:0/10217366 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed040bb2f0 con 0x7fed0414ae60 2024-03-18T15:34:15.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.896+0000 7fed02d9d640 1 --2- 172.21.15.5:0/10217366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed0414ae60 0x7fed040bbad0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fecfc00c770 tx=0x7fecfc00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:15.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.900+0000 7fecfaffd640 1 -- 172.21.15.5:0/10217366 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc003b80 con 0x7fed0414ae60 2024-03-18T15:34:15.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.900+0000 7fecfaffd640 1 -- 172.21.15.5:0/10217366 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecfc003d20 con 0x7fed0414ae60 2024-03-18T15:34:15.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.900+0000 7fed08a3f640 1 -- 172.21.15.5:0/10217366 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed04141f60 con 0x7fed0414ae60 2024-03-18T15:34:15.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.900+0000 7fecfaffd640 1 -- 172.21.15.5:0/10217366 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc00aa90 con 0x7fed0414ae60 2024-03-18T15:34:15.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.900+0000 7fed08a3f640 1 -- 172.21.15.5:0/10217366 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed041e32c0 con 0x7fed0414ae60 2024-03-18T15:34:15.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.900+0000 7fed08a3f640 1 -- 172.21.15.5:0/10217366 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed040bd100 con 0x7fed0414ae60 2024-03-18T15:34:15.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.900+0000 7fecfaffd640 1 -- 172.21.15.5:0/10217366 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fecfc00ac30 con 0x7fed0414ae60 2024-03-18T15:34:15.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.900+0000 7fecfaffd640 1 --2- 172.21.15.5:0/10217366 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fece4067660 0x7fece4069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:15.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.900+0000 7fed0259c640 1 --2- 172.21.15.5:0/10217366 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fece4067660 0x7fece4069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:15.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.900+0000 7fecfaffd640 1 -- 172.21.15.5:0/10217366 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 4943+0+0 (secure 0 0 0) 0x7fecfc085470 con 0x7fed0414ae60 2024-03-18T15:34:15.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.900+0000 7fed0259c640 1 --2- 172.21.15.5:0/10217366 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fece4067660 0x7fece4069b20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fecec009890 tx=0x7fecec008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:15.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:15.904+0000 7fecfaffd640 1 -- 172.21.15.5:0/10217366 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fecfc056480 con 0x7fed0414ae60 2024-03-18T15:34:16.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.080+0000 7fed08a3f640 1 -- 172.21.15.5:0/10217366 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fed04074a90 con 0x7fed0414ae60 2024-03-18T15:34:16.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.692+0000 7fecfaffd640 1 -- 172.21.15.5:0/10217366 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v26) ==== 97+0+0 (secure 0 0 0) 0x7fed04074a90 con 0x7fed0414ae60 2024-03-18T15:34:16.695 INFO:teuthology.orchestra.run.smithi005.stderr: removed all blocklist entries 2024-03-18T15:34:16.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.696+0000 7fed08a3f640 1 -- 172.21.15.5:0/10217366 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fece4067660 msgr2=0x7fece4069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:16.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.696+0000 7fed08a3f640 1 --2- 172.21.15.5:0/10217366 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fece4067660 0x7fece4069b20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fecec009890 tx=0x7fecec008040 comp rx=0 tx=0).stop 2024-03-18T15:34:16.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.696+0000 7fed08a3f640 1 -- 172.21.15.5:0/10217366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed0414ae60 msgr2=0x7fed040bbad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:16.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.696+0000 7fed08a3f640 1 --2- 172.21.15.5:0/10217366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed0414ae60 0x7fed040bbad0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fecfc00c770 tx=0x7fecfc00cc30 comp rx=0 tx=0).stop 2024-03-18T15:34:16.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.696+0000 7fed08a3f640 1 -- 172.21.15.5:0/10217366 shutdown_connections 2024-03-18T15:34:16.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.696+0000 7fed08a3f640 1 --2- 172.21.15.5:0/10217366 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fece4067660 0x7fece4069b20 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:16.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.696+0000 7fed08a3f640 1 --2- 172.21.15.5:0/10217366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed0414ae60 0x7fed040bbad0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:16.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.696+0000 7fed08a3f640 1 --2- 172.21.15.5:0/10217366 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fed04141430 0x7fed04139460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:16.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.696+0000 7fed08a3f640 1 --2- 172.21.15.5:0/10217366 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fed040b5710 0x7fed040bd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:16.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.696+0000 7fed08a3f640 1 -- 172.21.15.5:0/10217366 >> 172.21.15.5:0/10217366 conn(0x7fed040673c0 msgr2=0x7fed04148700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:16.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.696+0000 7fed08a3f640 1 -- 172.21.15.5:0/10217366 shutdown_connections 2024-03-18T15:34:16.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.696+0000 7fed08a3f640 1 -- 172.21.15.5:0/10217366 wait complete. 2024-03-18T15:34:16.721 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-18T15:34:16.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.844+0000 7fdd4fe5e640 1 Processor -- start 2024-03-18T15:34:16.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.844+0000 7fdd4fe5e640 1 -- start start 2024-03-18T15:34:16.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.844+0000 7fdd4fe5e640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fdd4814c410 0x7fdd480b2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:16.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.844+0000 7fdd4fe5e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fdd480b4780 0x7fdd480b2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:16.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.844+0000 7fdd4fe5e640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd4806d6b0 con 0x7fdd4806d2d0 2024-03-18T15:34:16.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.844+0000 7fdd4fe5e640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fdd480aeb70 con 0x7fdd480b4780 2024-03-18T15:34:16.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.844+0000 7fdd4fe5e640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fdd480ae520 con 0x7fdd4814c410 2024-03-18T15:34:16.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.844+0000 7fdd4e3fb640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fdd480b4780 0x7fdd480b2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:16.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.844+0000 7fdd4e3fb640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fdd480b4780 0x7fdd480b2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45904/0 (socket says 172.21.15.5:45904) 2024-03-18T15:34:16.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.844+0000 7fdd4e3fb640 1 -- 172.21.15.5:0/1911477827 learned_addr learned my addr 172.21.15.5:0/1911477827 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:16.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.844+0000 7fdd4d3f9640 1 --2- 172.21.15.5:0/1911477827 >> v2:172.21.15.80:3300/0 conn(0x7fdd4814c410 0x7fdd480b2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:16.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.844+0000 7fdd4e3fb640 1 -- 172.21.15.5:0/1911477827 >> v2:172.21.15.80:3300/0 conn(0x7fdd4814c410 msgr2=0x7fdd480b2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:16.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.844+0000 7fdd4e3fb640 1 --2- 172.21.15.5:0/1911477827 >> v2:172.21.15.80:3300/0 conn(0x7fdd4814c410 0x7fdd480b2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:16.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.844+0000 7fdd4e3fb640 1 -- 172.21.15.5:0/1911477827 >> v1:172.21.15.80:6789/0 conn(0x7fdd4806d2d0 legacy=0x7fdd4814bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:16.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.844+0000 7fdd4e3fb640 1 -- 172.21.15.5:0/1911477827 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd480ae800 con 0x7fdd480b4780 2024-03-18T15:34:16.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.844+0000 7fdd4d3f9640 1 --2- 172.21.15.5:0/1911477827 >> v2:172.21.15.80:3300/0 conn(0x7fdd4814c410 0x7fdd480b2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:34:16.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.848+0000 7fdd4e3fb640 1 --2- 172.21.15.5:0/1911477827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd480b4780 0x7fdd480b2bd0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fdd480b53d0 tx=0x7fdd4001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95bd717739d5582f server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:16.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.848+0000 7fdd4cbf8640 1 -- 172.21.15.5:0/1911477827 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd400a5640 con 0x7fdd480b4780 2024-03-18T15:34:16.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.848+0000 7fdd4cbf8640 1 -- 172.21.15.5:0/1911477827 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd400a57e0 con 0x7fdd480b4780 2024-03-18T15:34:16.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.848+0000 7fdd4cbf8640 1 -- 172.21.15.5:0/1911477827 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd400a5bb0 con 0x7fdd480b4780 2024-03-18T15:34:16.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.848+0000 7fdd4fe5e640 1 -- 172.21.15.5:0/1911477827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd480b4780 msgr2=0x7fdd480b2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:16.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.848+0000 7fdd4fe5e640 1 --2- 172.21.15.5:0/1911477827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd480b4780 0x7fdd480b2bd0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fdd480b53d0 tx=0x7fdd4001c960 comp rx=0 tx=0).stop 2024-03-18T15:34:16.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.848+0000 7fdd4fe5e640 1 -- 172.21.15.5:0/1911477827 shutdown_connections 2024-03-18T15:34:16.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.848+0000 7fdd4fe5e640 1 --2- 172.21.15.5:0/1911477827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd480b4780 0x7fdd480b2bd0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:16.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.848+0000 7fdd4fe5e640 1 --2- 172.21.15.5:0/1911477827 >> v2:172.21.15.80:3300/0 conn(0x7fdd4814c410 0x7fdd480b2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:16.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.848+0000 7fdd4fe5e640 1 -- 172.21.15.5:0/1911477827 >> 172.21.15.5:0/1911477827 conn(0x7fdd480b5670 msgr2=0x7fdd480b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:16.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.848+0000 7fdd4fe5e640 1 -- 172.21.15.5:0/1911477827 shutdown_connections 2024-03-18T15:34:16.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.848+0000 7fdd4fe5e640 1 -- 172.21.15.5:0/1911477827 wait complete. 2024-03-18T15:34:16.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.848+0000 7fdd4fe5e640 1 Processor -- start 2024-03-18T15:34:16.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.848+0000 7fdd4fe5e640 1 -- start start 2024-03-18T15:34:16.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.852+0000 7fdd4fe5e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd4806d2d0 0x7fdd480b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:16.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.852+0000 7fdd4fe5e640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdd480b4780 0x7fdd481e7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:16.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.852+0000 7fdd4fe5e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd4814c410 0x7fdd480702f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:16.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.852+0000 7fdd4fe5e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdd48158bf0 con 0x7fdd4806d2d0 2024-03-18T15:34:16.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.852+0000 7fdd4fe5e640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fdd48158a70 con 0x7fdd4814c410 2024-03-18T15:34:16.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.852+0000 7fdd4fe5e640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fdd48158d70 con 0x7fdd480b4780 2024-03-18T15:34:16.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.852+0000 7fdd4dbfa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd4806d2d0 0x7fdd480b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:16.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.852+0000 7fdd4dbfa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd4806d2d0 0x7fdd480b7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45912/0 (socket says 172.21.15.5:45912) 2024-03-18T15:34:16.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.852+0000 7fdd4dbfa640 1 -- 172.21.15.5:0/1729508278 learned_addr learned my addr 172.21.15.5:0/1729508278 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:16.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.852+0000 7fdd4d3f9640 1 --2- 172.21.15.5:0/1729508278 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdd480b4780 0x7fdd481e7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:16.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.852+0000 7fdd4e3fb640 1 --2- 172.21.15.5:0/1729508278 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd4814c410 0x7fdd480702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:16.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.852+0000 7fdd4dbfa640 1 -- 172.21.15.5:0/1729508278 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdd480b4780 msgr2=0x7fdd481e7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:16.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.852+0000 7fdd4dbfa640 1 --2- 172.21.15.5:0/1729508278 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdd480b4780 0x7fdd481e7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:16.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.852+0000 7fdd4dbfa640 1 -- 172.21.15.5:0/1729508278 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd4814c410 msgr2=0x7fdd480702f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:16.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.852+0000 7fdd4dbfa640 1 --2- 172.21.15.5:0/1729508278 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd4814c410 0x7fdd480702f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:16.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.852+0000 7fdd4dbfa640 1 -- 172.21.15.5:0/1729508278 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd4814fd70 con 0x7fdd4806d2d0 2024-03-18T15:34:16.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.852+0000 7fdd4dbfa640 1 --2- 172.21.15.5:0/1729508278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd4806d2d0 0x7fdd480b7fe0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fdd3800c500 tx=0x7fdd3800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:16.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.852+0000 7fdd3e7fc640 1 -- 172.21.15.5:0/1729508278 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd38016020 con 0x7fdd4806d2d0 2024-03-18T15:34:16.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.852+0000 7fdd3e7fc640 1 -- 172.21.15.5:0/1729508278 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd38005220 con 0x7fdd4806d2d0 2024-03-18T15:34:16.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.852+0000 7fdd3e7fc640 1 -- 172.21.15.5:0/1729508278 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd38007030 con 0x7fdd4806d2d0 2024-03-18T15:34:16.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.852+0000 7fdd4fe5e640 1 -- 172.21.15.5:0/1729508278 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd4814d1a0 con 0x7fdd4806d2d0 2024-03-18T15:34:16.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.852+0000 7fdd4fe5e640 1 -- 172.21.15.5:0/1729508278 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd48071430 con 0x7fdd4806d2d0 2024-03-18T15:34:16.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.852+0000 7fdd4fe5e640 1 -- 172.21.15.5:0/1729508278 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd4804f550 con 0x7fdd4806d2d0 2024-03-18T15:34:16.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.852+0000 7fdd3e7fc640 1 -- 172.21.15.5:0/1729508278 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fdd380071d0 con 0x7fdd4806d2d0 2024-03-18T15:34:16.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.852+0000 7fdd3e7fc640 1 --2- 172.21.15.5:0/1729508278 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fdd2c067660 0x7fdd2c069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:16.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.856+0000 7fdd3e7fc640 1 -- 172.21.15.5:0/1729508278 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 4857+0+0 (secure 0 0 0) 0x7fdd38084f80 con 0x7fdd4806d2d0 2024-03-18T15:34:16.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.856+0000 7fdd4d3f9640 1 --2- 172.21.15.5:0/1729508278 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fdd2c067660 0x7fdd2c069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:16.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.856+0000 7fdd4d3f9640 1 --2- 172.21.15.5:0/1729508278 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fdd2c067660 0x7fdd2c069b20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdd4814beb0 tx=0x7fdd44008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:16.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:16.860+0000 7fdd3e7fc640 1 -- 172.21.15.5:0/1729508278 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fdd38020020 con 0x7fdd4806d2d0 2024-03-18T15:34:17.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.076+0000 7fdd4fe5e640 1 -- 172.21.15.5:0/1729508278 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fdd480b1c80 con 0x7fdd4806d2d0 2024-03-18T15:34:17.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.076+0000 7fdd3e7fc640 1 -- 172.21.15.5:0/1729508278 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) ==== 80+0+5207 (secure 0 0 0) 0x7fdd380074c0 con 0x7fdd4806d2d0 2024-03-18T15:34:17.079 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:34:17.079 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-03-18T15:34:17.079 INFO:teuthology.orchestra.run.smithi005.stdout: "auth_dump": [ 2024-03-18T15:34:17.080 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-03-18T15:34:17.080 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.a", 2024-03-18T15:34:17.080 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCeXvhllrlWLRAAeoM9rVm/NIT0Z1HQC7hlrg==", 2024-03-18T15:34:17.080 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-03-18T15:34:17.080 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-03-18T15:34:17.080 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-03-18T15:34:17.080 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-03-18T15:34:17.080 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-03-18T15:34:17.080 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-03-18T15:34:17.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-03-18T15:34:17.080 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-03-18T15:34:17.080 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mds.b", 2024-03-18T15:34:17.080 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCeXvhlp5F9MxAAParGkBY7Sc9gOUKZVLxBzg==", 2024-03-18T15:34:17.080 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-03-18T15:34:17.080 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-03-18T15:34:17.081 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-03-18T15:34:17.081 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-03-18T15:34:17.081 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-03-18T15:34:17.081 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-03-18T15:34:17.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-03-18T15:34:17.081 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-03-18T15:34:17.081 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.0", 2024-03-18T15:34:17.081 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCgXvhlJ3S2BBAAX7b5T6UKtrnOhSIhy2V5nw==", 2024-03-18T15:34:17.081 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-03-18T15:34:17.081 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-03-18T15:34:17.081 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-03-18T15:34:17.081 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-03-18T15:34:17.081 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-03-18T15:34:17.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-03-18T15:34:17.082 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-03-18T15:34:17.082 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.1", 2024-03-18T15:34:17.082 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCjXvhl/y82CBAAJXuiwwphz1dLCegtoBw/sQ==", 2024-03-18T15:34:17.082 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-03-18T15:34:17.082 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-03-18T15:34:17.082 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-03-18T15:34:17.082 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-03-18T15:34:17.082 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-03-18T15:34:17.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-03-18T15:34:17.082 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-03-18T15:34:17.082 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.2", 2024-03-18T15:34:17.082 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCmXvhlU+yBDRAA9DtMz35eiXflG1zQuqEScg==", 2024-03-18T15:34:17.082 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-03-18T15:34:17.082 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-03-18T15:34:17.083 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-03-18T15:34:17.083 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-03-18T15:34:17.083 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-03-18T15:34:17.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-03-18T15:34:17.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-03-18T15:34:17.083 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.3", 2024-03-18T15:34:17.083 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCpXvhlsyq3ExAAY53wP1De3pPBaZuKF7kFrw==", 2024-03-18T15:34:17.083 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-03-18T15:34:17.083 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-03-18T15:34:17.083 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-03-18T15:34:17.083 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-03-18T15:34:17.083 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-03-18T15:34:17.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-03-18T15:34:17.083 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-03-18T15:34:17.084 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.4", 2024-03-18T15:34:17.084 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCvXvhlwwEnEBAAS9wCX93LDdiGtiyKWy/1lA==", 2024-03-18T15:34:17.084 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-03-18T15:34:17.084 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-03-18T15:34:17.084 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-03-18T15:34:17.084 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-03-18T15:34:17.084 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-03-18T15:34:17.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-03-18T15:34:17.084 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-03-18T15:34:17.084 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.5", 2024-03-18T15:34:17.084 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCyXvhlxC7HFBAACkH4xW+XsH+LMc384ulWow==", 2024-03-18T15:34:17.084 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-03-18T15:34:17.084 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-03-18T15:34:17.084 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-03-18T15:34:17.085 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-03-18T15:34:17.085 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-03-18T15:34:17.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-03-18T15:34:17.085 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-03-18T15:34:17.085 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.6", 2024-03-18T15:34:17.085 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQC1XvhlgHeLGxAA2Q5gGrjSpaVmIdwunlRBBw==", 2024-03-18T15:34:17.085 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-03-18T15:34:17.085 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-03-18T15:34:17.085 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-03-18T15:34:17.085 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-03-18T15:34:17.085 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-03-18T15:34:17.086 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-03-18T15:34:17.086 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-03-18T15:34:17.086 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "osd.7", 2024-03-18T15:34:17.086 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQC4XvhlRB7XIRAAqxxe4tWUrF3n5VK/O4PHcg==", 2024-03-18T15:34:17.086 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-03-18T15:34:17.086 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow profile osd", 2024-03-18T15:34:17.086 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile osd", 2024-03-18T15:34:17.086 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-03-18T15:34:17.086 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-03-18T15:34:17.086 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-03-18T15:34:17.086 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-03-18T15:34:17.086 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.0", 2024-03-18T15:34:17.086 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCeXvhlDBsBORAA9deuFWz05eGfJp1vj5Vlkg==", 2024-03-18T15:34:17.087 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-03-18T15:34:17.087 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow", 2024-03-18T15:34:17.087 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow r", 2024-03-18T15:34:17.087 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow rw", 2024-03-18T15:34:17.087 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow rwx" 2024-03-18T15:34:17.087 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-03-18T15:34:17.087 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-03-18T15:34:17.087 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-03-18T15:34:17.087 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.admin", 2024-03-18T15:34:17.087 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCeXvhlN/jCExAAi6VG7YUg32Ftv3KUuQ3Pxg==", 2024-03-18T15:34:17.087 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-03-18T15:34:17.087 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-03-18T15:34:17.087 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr": "allow *", 2024-03-18T15:34:17.087 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow *", 2024-03-18T15:34:17.088 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-03-18T15:34:17.088 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-03-18T15:34:17.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-03-18T15:34:17.088 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-03-18T15:34:17.088 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.bootstrap-mds", 2024-03-18T15:34:17.088 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQC+XvhlwnE6BRAACPxFhuADOTD7jY4QvCsEkw==", 2024-03-18T15:34:17.088 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-03-18T15:34:17.088 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile bootstrap-mds" 2024-03-18T15:34:17.088 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-03-18T15:34:17.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-03-18T15:34:17.088 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-03-18T15:34:17.088 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.bootstrap-mgr", 2024-03-18T15:34:17.088 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQC+XvhlvY06BRAAD2ai9MaY2grzPGCGGqwgDA==", 2024-03-18T15:34:17.088 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-03-18T15:34:17.089 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile bootstrap-mgr" 2024-03-18T15:34:17.089 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-03-18T15:34:17.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-03-18T15:34:17.089 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-03-18T15:34:17.089 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.bootstrap-osd", 2024-03-18T15:34:17.089 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQC+XvhlYaM6BRAAKOciuQnSSBSfs7INhkQgfA==", 2024-03-18T15:34:17.089 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-03-18T15:34:17.089 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile bootstrap-osd" 2024-03-18T15:34:17.089 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-03-18T15:34:17.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-03-18T15:34:17.089 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-03-18T15:34:17.089 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.bootstrap-rbd", 2024-03-18T15:34:17.089 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQC+XvhlaLg6BRAA4FFYkZGx1DK3AWKjrMhAFA==", 2024-03-18T15:34:17.089 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-03-18T15:34:17.089 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile bootstrap-rbd" 2024-03-18T15:34:17.090 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-03-18T15:34:17.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-03-18T15:34:17.090 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-03-18T15:34:17.090 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-03-18T15:34:17.090 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQC+XvhlR9I6BRAAIEWSu0wxNyNzo3CiSMhenw==", 2024-03-18T15:34:17.090 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-03-18T15:34:17.090 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-03-18T15:34:17.090 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-03-18T15:34:17.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-03-18T15:34:17.090 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-03-18T15:34:17.090 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "client.bootstrap-rgw", 2024-03-18T15:34:17.090 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQC+Xvhl9ug6BRAAY3wHUsTRmzDFhYnK2wBfnw==", 2024-03-18T15:34:17.090 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-03-18T15:34:17.091 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile bootstrap-rgw" 2024-03-18T15:34:17.091 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-03-18T15:34:17.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-03-18T15:34:17.091 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-03-18T15:34:17.091 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mgr.x", 2024-03-18T15:34:17.091 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCeXvhlF4OfKhAAost72x1wdCRqzhgIdqEw1w==", 2024-03-18T15:34:17.091 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-03-18T15:34:17.091 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-03-18T15:34:17.091 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile mgr", 2024-03-18T15:34:17.091 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-03-18T15:34:17.091 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-03-18T15:34:17.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-03-18T15:34:17.091 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-03-18T15:34:17.091 INFO:teuthology.orchestra.run.smithi005.stdout: "entity": "mgr.y", 2024-03-18T15:34:17.092 INFO:teuthology.orchestra.run.smithi005.stdout: "key": "AQCeXvhlZ1/DJxAAhLPs51ARBSnFH+qu0h+1Jw==", 2024-03-18T15:34:17.092 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": { 2024-03-18T15:34:17.092 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": "allow *", 2024-03-18T15:34:17.092 INFO:teuthology.orchestra.run.smithi005.stdout: "mon": "allow profile mgr", 2024-03-18T15:34:17.092 INFO:teuthology.orchestra.run.smithi005.stdout: "osd": "allow *" 2024-03-18T15:34:17.092 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-03-18T15:34:17.092 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-03-18T15:34:17.092 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-03-18T15:34:17.092 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-03-18T15:34:17.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.076+0000 7fdd4fe5e640 1 -- 172.21.15.5:0/1729508278 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fdd2c067660 msgr2=0x7fdd2c069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:17.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.076+0000 7fdd4fe5e640 1 --2- 172.21.15.5:0/1729508278 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fdd2c067660 0x7fdd2c069b20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdd4814beb0 tx=0x7fdd44008040 comp rx=0 tx=0).stop 2024-03-18T15:34:17.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.080+0000 7fdd4fe5e640 1 -- 172.21.15.5:0/1729508278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd4806d2d0 msgr2=0x7fdd480b7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:17.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.080+0000 7fdd4fe5e640 1 --2- 172.21.15.5:0/1729508278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd4806d2d0 0x7fdd480b7fe0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fdd3800c500 tx=0x7fdd3800c9c0 comp rx=0 tx=0).stop 2024-03-18T15:34:17.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.080+0000 7fdd4fe5e640 1 -- 172.21.15.5:0/1729508278 shutdown_connections 2024-03-18T15:34:17.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.080+0000 7fdd4fe5e640 1 --2- 172.21.15.5:0/1729508278 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fdd2c067660 0x7fdd2c069b20 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:17.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.080+0000 7fdd4fe5e640 1 --2- 172.21.15.5:0/1729508278 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd4814c410 0x7fdd480702f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:17.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.080+0000 7fdd4fe5e640 1 --2- 172.21.15.5:0/1729508278 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdd480b4780 0x7fdd481e7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:17.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.080+0000 7fdd4fe5e640 1 --2- 172.21.15.5:0/1729508278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd4806d2d0 0x7fdd480b7fe0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:17.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.080+0000 7fdd4fe5e640 1 -- 172.21.15.5:0/1729508278 >> 172.21.15.5:0/1729508278 conn(0x7fdd480b5670 msgr2=0x7fdd480b0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:17.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.080+0000 7fdd4fe5e640 1 -- 172.21.15.5:0/1729508278 shutdown_connections 2024-03-18T15:34:17.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.080+0000 7fdd4fe5e640 1 -- 172.21.15.5:0/1729508278 wait complete. 2024-03-18T15:34:17.098 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-03-18T15:34:17.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.216+0000 7f79e3f96640 1 Processor -- start 2024-03-18T15:34:17.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.216+0000 7f79e3f96640 1 -- start start 2024-03-18T15:34:17.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.216+0000 7f79e3f96640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f79dc072b40 0x7f79dc06d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:17.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.216+0000 7f79e3f96640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79dc0afb70 con 0x7f79dc0b26a0 2024-03-18T15:34:17.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.216+0000 7f79e3f96640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79dc0b4a20 con 0x7f79dc0af790 2024-03-18T15:34:17.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.216+0000 7f79e3f96640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f79dc06d5b0 con 0x7f79dc072b40 2024-03-18T15:34:17.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.216+0000 7f79e1531640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f79dc072b40 0x7f79dc06d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:17.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.216+0000 7f79e1531640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f79dc072b40 0x7f79dc06d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45926/0 (socket says 172.21.15.5:45926) 2024-03-18T15:34:17.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.216+0000 7f79e1531640 1 -- 172.21.15.5:0/1413075290 learned_addr learned my addr 172.21.15.5:0/1413075290 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:17.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.216+0000 7f79e2533640 1 --1- 172.21.15.5:0/1413075290 >> v1:172.21.15.5:6789/0 conn(0x7f79dc0b26a0 0x7f79dc0b2af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51550/0 (socket says 172.21.15.5:51550) 2024-03-18T15:34:17.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.216+0000 7f79e1531640 1 -- 172.21.15.5:0/1413075290 >> v1:172.21.15.80:6789/0 conn(0x7f79dc0af790 legacy=0x7f79dc147780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:17.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.216+0000 7f79e1531640 1 -- 172.21.15.5:0/1413075290 >> v1:172.21.15.5:6789/0 conn(0x7f79dc0b26a0 legacy=0x7f79dc0b2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:17.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.216+0000 7f79e1531640 1 -- 172.21.15.5:0/1413075290 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79dc0b4d00 con 0x7f79dc072b40 2024-03-18T15:34:17.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.220+0000 7f79e1531640 1 --2- 172.21.15.5:0/1413075290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79dc072b40 0x7f79dc06d070 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f79cc009870 tx=0x7f79cc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef77194e2a640250 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:17.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.220+0000 7f79e0d30640 1 -- 172.21.15.5:0/1413075290 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79cc0b1070 con 0x7f79dc072b40 2024-03-18T15:34:17.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.220+0000 7f79e0d30640 1 -- 172.21.15.5:0/1413075290 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79cc0ac600 con 0x7f79dc072b40 2024-03-18T15:34:17.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.220+0000 7f79e0d30640 1 -- 172.21.15.5:0/1413075290 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79cc0ac9d0 con 0x7f79dc072b40 2024-03-18T15:34:17.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.220+0000 7f79e3f96640 1 -- 172.21.15.5:0/1413075290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79dc072b40 msgr2=0x7f79dc06d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:17.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.220+0000 7f79e3f96640 1 --2- 172.21.15.5:0/1413075290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79dc072b40 0x7f79dc06d070 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f79cc009870 tx=0x7f79cc01c490 comp rx=0 tx=0).stop 2024-03-18T15:34:17.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.220+0000 7f79e3f96640 1 -- 172.21.15.5:0/1413075290 shutdown_connections 2024-03-18T15:34:17.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.220+0000 7f79e3f96640 1 --2- 172.21.15.5:0/1413075290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79dc072b40 0x7f79dc06d070 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:17.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.220+0000 7f79e3f96640 1 -- 172.21.15.5:0/1413075290 >> 172.21.15.5:0/1413075290 conn(0x7f79dc0b5670 msgr2=0x7f79dc0b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:17.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.220+0000 7f79e3f96640 1 -- 172.21.15.5:0/1413075290 shutdown_connections 2024-03-18T15:34:17.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.220+0000 7f79e3f96640 1 -- 172.21.15.5:0/1413075290 wait complete. 2024-03-18T15:34:17.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.220+0000 7f79e3f96640 1 Processor -- start 2024-03-18T15:34:17.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.220+0000 7f79e3f96640 1 -- start start 2024-03-18T15:34:17.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.220+0000 7f79e3f96640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f79dc072b40 0x7f79dc071710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:17.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.220+0000 7f79e3f96640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79dc0af790 0x7f79dc13f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:17.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.220+0000 7f79e3f96640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79dc0b26a0 0x7f79dc138470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:17.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.220+0000 7f79e3f96640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f79dc152ac0 con 0x7f79dc0b26a0 2024-03-18T15:34:17.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.220+0000 7f79e3f96640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f79dc152940 con 0x7f79dc0af790 2024-03-18T15:34:17.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.220+0000 7f79e3f96640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f79dc152c40 con 0x7f79dc072b40 2024-03-18T15:34:17.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.220+0000 7f79e2533640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79dc0b26a0 0x7f79dc138470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:17.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.220+0000 7f79e2533640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79dc0b26a0 0x7f79dc138470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45934/0 (socket says 172.21.15.5:45934) 2024-03-18T15:34:17.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.220+0000 7f79e2533640 1 -- 172.21.15.5:0/590728454 learned_addr learned my addr 172.21.15.5:0/590728454 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:17.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.220+0000 7f79e1d32640 1 --2- 172.21.15.5:0/590728454 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f79dc072b40 0x7f79dc071710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:17.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.220+0000 7f79e1531640 1 --2- 172.21.15.5:0/590728454 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79dc0af790 0x7f79dc13f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:17.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.220+0000 7f79e2533640 1 -- 172.21.15.5:0/590728454 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f79dc072b40 msgr2=0x7f79dc071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:17.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.220+0000 7f79e2533640 1 --2- 172.21.15.5:0/590728454 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f79dc072b40 0x7f79dc071710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:17.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.220+0000 7f79e2533640 1 -- 172.21.15.5:0/590728454 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79dc0af790 msgr2=0x7f79dc13f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:17.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.220+0000 7f79e2533640 1 --2- 172.21.15.5:0/590728454 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79dc0af790 0x7f79dc13f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:17.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.220+0000 7f79e2533640 1 -- 172.21.15.5:0/590728454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79dc13f860 con 0x7f79dc0b26a0 2024-03-18T15:34:17.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.220+0000 7f79e1531640 1 --2- 172.21.15.5:0/590728454 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79dc0af790 0x7f79dc13f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:17.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.220+0000 7f79e2533640 1 --2- 172.21.15.5:0/590728454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79dc0b26a0 0x7f79dc138470 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f79d400c700 tx=0x7f79d400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:17.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.220+0000 7f79d27fc640 1 -- 172.21.15.5:0/590728454 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d4017070 con 0x7f79dc0b26a0 2024-03-18T15:34:17.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.220+0000 7f79d27fc640 1 -- 172.21.15.5:0/590728454 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79d400ce10 con 0x7f79dc0b26a0 2024-03-18T15:34:17.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.220+0000 7f79d27fc640 1 -- 172.21.15.5:0/590728454 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d400a7e0 con 0x7f79dc0b26a0 2024-03-18T15:34:17.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.224+0000 7f79e3f96640 1 -- 172.21.15.5:0/590728454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79dc13fb40 con 0x7f79dc0b26a0 2024-03-18T15:34:17.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.224+0000 7f79e3f96640 1 -- 172.21.15.5:0/590728454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79dc139560 con 0x7f79dc0b26a0 2024-03-18T15:34:17.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.224+0000 7f79e3f96640 1 -- 172.21.15.5:0/590728454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79dc135a10 con 0x7f79dc0b26a0 2024-03-18T15:34:17.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.224+0000 7f79d27fc640 1 -- 172.21.15.5:0/590728454 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f79d401a070 con 0x7f79dc0b26a0 2024-03-18T15:34:17.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.224+0000 7f79d27fc640 1 --2- 172.21.15.5:0/590728454 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f79c0067660 0x7f79c0069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:17.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.224+0000 7f79d27fc640 1 -- 172.21.15.5:0/590728454 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 4857+0+0 (secure 0 0 0) 0x7f79d4084940 con 0x7f79dc0b26a0 2024-03-18T15:34:17.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.224+0000 7f79e1d32640 1 --2- 172.21.15.5:0/590728454 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f79c0067660 0x7f79c0069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:17.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.224+0000 7f79e1d32640 1 --2- 172.21.15.5:0/590728454 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f79c0067660 0x7f79c0069b20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f79dc145010 tx=0x7f79d8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:17.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.228+0000 7f79d27fc640 1 -- 172.21.15.5:0/590728454 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f79dc135a10 con 0x7f79dc0b26a0 2024-03-18T15:34:17.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.448+0000 7f79e3f96640 1 -- 172.21.15.5:0/590728454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f79dc04f550 con 0x7f79dc0b26a0 2024-03-18T15:34:17.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.452+0000 7f79d27fc640 1 -- 172.21.15.5:0/590728454 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7f79d4055350 con 0x7f79dc0b26a0 2024-03-18T15:34:17.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.456+0000 7f79e3f96640 1 -- 172.21.15.5:0/590728454 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f79c0067660 msgr2=0x7f79c0069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:17.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.456+0000 7f79e3f96640 1 --2- 172.21.15.5:0/590728454 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f79c0067660 0x7f79c0069b20 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f79dc145010 tx=0x7f79d8009040 comp rx=0 tx=0).stop 2024-03-18T15:34:17.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.456+0000 7f79e3f96640 1 -- 172.21.15.5:0/590728454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79dc0b26a0 msgr2=0x7f79dc138470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:17.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.456+0000 7f79e3f96640 1 --2- 172.21.15.5:0/590728454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79dc0b26a0 0x7f79dc138470 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f79d400c700 tx=0x7f79d400cbc0 comp rx=0 tx=0).stop 2024-03-18T15:34:17.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.456+0000 7f79e3f96640 1 -- 172.21.15.5:0/590728454 shutdown_connections 2024-03-18T15:34:17.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.456+0000 7f79e3f96640 1 --2- 172.21.15.5:0/590728454 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f79c0067660 0x7f79c0069b20 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:17.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.456+0000 7f79e3f96640 1 --2- 172.21.15.5:0/590728454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79dc0b26a0 0x7f79dc138470 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:17.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.456+0000 7f79e3f96640 1 --2- 172.21.15.5:0/590728454 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79dc0af790 0x7f79dc13f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:17.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.456+0000 7f79e3f96640 1 --2- 172.21.15.5:0/590728454 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f79dc072b40 0x7f79dc071710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:17.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.456+0000 7f79e3f96640 1 -- 172.21.15.5:0/590728454 >> 172.21.15.5:0/590728454 conn(0x7f79dc0b5670 msgr2=0x7f79dc06c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:17.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.456+0000 7f79e3f96640 1 -- 172.21.15.5:0/590728454 shutdown_connections 2024-03-18T15:34:17.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.456+0000 7f79e3f96640 1 -- 172.21.15.5:0/590728454 wait complete. 2024-03-18T15:34:17.475 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-03-18T15:34:17.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.600+0000 7fa47f0ac640 1 Processor -- start 2024-03-18T15:34:17.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.600+0000 7fa47f0ac640 1 -- start start 2024-03-18T15:34:17.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.604+0000 7fa47f0ac640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa4780b5710 0x7fa47814a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:17.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.604+0000 7fa47f0ac640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4780b9d50 con 0x7fa47814ae60 2024-03-18T15:34:17.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.604+0000 7fa47f0ac640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4780b4c30 con 0x7fa478141600 2024-03-18T15:34:17.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.604+0000 7fa47f0ac640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fa478141a90 con 0x7fa4780b5710 2024-03-18T15:34:17.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.604+0000 7fa477fff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa47814ae60 0x7fa478140ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51552/0 (socket says 172.21.15.5:51552) 2024-03-18T15:34:17.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.604+0000 7fa477fff640 1 -- 172.21.15.5:0/1103679015 learned_addr learned my addr 172.21.15.5:0/1103679015 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:17.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.604+0000 7fa47ce48640 1 --2- 172.21.15.5:0/1103679015 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa4780b5710 0x7fa47814a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:17.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.604+0000 7fa4777fe640 1 -- 172.21.15.5:0/1103679015 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2018246923 0 0) 0x7fa4780b9d50 con 0x7fa47814ae60 2024-03-18T15:34:17.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.604+0000 7fa4777fe640 1 -- 172.21.15.5:0/1103679015 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa460003530 con 0x7fa47814ae60 2024-03-18T15:34:17.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.604+0000 7fa4777fe640 1 -- 172.21.15.5:0/1103679015 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 767343232 0 0) 0x7fa460003530 con 0x7fa47814ae60 2024-03-18T15:34:17.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.604+0000 7fa4777fe640 1 -- 172.21.15.5:0/1103679015 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa4780b5710 msgr2=0x7fa47814a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:17.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.604+0000 7fa4777fe640 1 --2- 172.21.15.5:0/1103679015 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa4780b5710 0x7fa47814a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:17.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.604+0000 7fa4777fe640 1 -- 172.21.15.5:0/1103679015 >> v1:172.21.15.80:6789/0 conn(0x7fa478141600 legacy=0x7fa478151cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:17.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.604+0000 7fa4777fe640 1 -- 172.21.15.5:0/1103679015 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4780b9210 con 0x7fa47814ae60 2024-03-18T15:34:17.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.604+0000 7fa4777fe640 1 -- 172.21.15.5:0/1103679015 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3837549003 0 0) 0x7fa46c001680 con 0x7fa47814ae60 2024-03-18T15:34:17.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.604+0000 7fa4777fe640 1 -- 172.21.15.5:0/1103679015 >> v1:172.21.15.5:6789/0 conn(0x7fa47814ae60 legacy=0x7fa478140ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:17.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.604+0000 7fa4777fe640 1 --2- 172.21.15.5:0/1103679015 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa4780b5710 0x7fa460003ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:17.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.604+0000 7fa4777fe640 1 --2- 172.21.15.5:0/1103679015 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa4600040a0 0x7fa460004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:17.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.604+0000 7fa4777fe640 1 --2- 172.21.15.5:0/1103679015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa460004e70 0x7fa46000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:17.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.604+0000 7fa4777fe640 1 -- 172.21.15.5:0/1103679015 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa460005660 con 0x7fa460004e70 2024-03-18T15:34:17.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.604+0000 7fa4777fe640 1 -- 172.21.15.5:0/1103679015 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa46000bf10 con 0x7fa4600040a0 2024-03-18T15:34:17.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.604+0000 7fa4777fe640 1 -- 172.21.15.5:0/1103679015 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fa46000c0c0 con 0x7fa4780b5710 2024-03-18T15:34:17.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.604+0000 7fa47d649640 1 --2- 172.21.15.5:0/1103679015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa460004e70 0x7fa46000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:17.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.604+0000 7fa47ce48640 1 --2- 172.21.15.5:0/1103679015 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa4780b5710 0x7fa460003ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:17.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.604+0000 7fa477fff640 1 --2- 172.21.15.5:0/1103679015 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa4600040a0 0x7fa460004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:17.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.604+0000 7fa47d649640 1 -- 172.21.15.5:0/1103679015 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa4780b5710 msgr2=0x7fa460003ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:17.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.604+0000 7fa47d649640 1 --2- 172.21.15.5:0/1103679015 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa4780b5710 0x7fa460003ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:17.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.604+0000 7fa47d649640 1 -- 172.21.15.5:0/1103679015 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa4600040a0 msgr2=0x7fa460004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:17.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.604+0000 7fa47d649640 1 --2- 172.21.15.5:0/1103679015 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa4600040a0 0x7fa460004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:17.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.604+0000 7fa47d649640 1 -- 172.21.15.5:0/1103679015 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa460003530 con 0x7fa460004e70 2024-03-18T15:34:17.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.604+0000 7fa477fff640 1 --2- 172.21.15.5:0/1103679015 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa4600040a0 0x7fa460004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:34:17.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.608+0000 7fa47d649640 1 --2- 172.21.15.5:0/1103679015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa460004e70 0x7fa46000b9d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa470001e70 tx=0x7fa47001c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bda36df8f28704b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:17.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.608+0000 7fa4777fe640 1 -- 172.21.15.5:0/1103679015 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4700af070 con 0x7fa460004e70 2024-03-18T15:34:17.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.608+0000 7fa4777fe640 1 -- 172.21.15.5:0/1103679015 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa470003b80 con 0x7fa460004e70 2024-03-18T15:34:17.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.608+0000 7fa4777fe640 1 -- 172.21.15.5:0/1103679015 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa470001a90 con 0x7fa460004e70 2024-03-18T15:34:17.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.608+0000 7fa47f0ac640 1 -- 172.21.15.5:0/1103679015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa460004e70 msgr2=0x7fa46000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:17.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.608+0000 7fa47f0ac640 1 --2- 172.21.15.5:0/1103679015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa460004e70 0x7fa46000b9d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa470001e70 tx=0x7fa47001c720 comp rx=0 tx=0).stop 2024-03-18T15:34:17.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.608+0000 7fa47d649640 1 -- 172.21.15.5:0/1103679015 reap_dead start 2024-03-18T15:34:17.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.608+0000 7fa47f0ac640 1 -- 172.21.15.5:0/1103679015 shutdown_connections 2024-03-18T15:34:17.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.608+0000 7fa47f0ac640 1 -- 172.21.15.5:0/1103679015 >> 172.21.15.5:0/1103679015 conn(0x7fa4780673c0 msgr2=0x7fa4780677d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:17.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.608+0000 7fa47f0ac640 1 -- 172.21.15.5:0/1103679015 shutdown_connections 2024-03-18T15:34:17.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.608+0000 7fa47f0ac640 1 -- 172.21.15.5:0/1103679015 wait complete. 2024-03-18T15:34:17.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.608+0000 7fa47f0ac640 1 Processor -- start 2024-03-18T15:34:17.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.608+0000 7fa47f0ac640 1 -- start start 2024-03-18T15:34:17.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.608+0000 7fa47f0ac640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa460004e70 0x7fa478149ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:17.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.608+0000 7fa47f0ac640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa47814dd70 0x7fa47814e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:17.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.608+0000 7fa47f0ac640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa47813bea0 0x7fa47813c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:17.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.608+0000 7fa47f0ac640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa4780ba0c0 con 0x7fa460004e70 2024-03-18T15:34:17.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.608+0000 7fa47f0ac640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa4780b4570 con 0x7fa47813bea0 2024-03-18T15:34:17.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.608+0000 7fa47f0ac640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fa4780b6940 con 0x7fa47814dd70 2024-03-18T15:34:17.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.608+0000 7fa47ce48640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa460004e70 0x7fa478149ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:17.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.608+0000 7fa47ce48640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa460004e70 0x7fa478149ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45946/0 (socket says 172.21.15.5:45946) 2024-03-18T15:34:17.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.608+0000 7fa47ce48640 1 -- 172.21.15.5:0/4237772485 learned_addr learned my addr 172.21.15.5:0/4237772485 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:17.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.608+0000 7fa477fff640 1 --2- 172.21.15.5:0/4237772485 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa47814dd70 0x7fa47814e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:17.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.608+0000 7fa47d649640 1 --2- 172.21.15.5:0/4237772485 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa47813bea0 0x7fa47813c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:17.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.608+0000 7fa47ce48640 1 -- 172.21.15.5:0/4237772485 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa47814dd70 msgr2=0x7fa47814e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:17.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.608+0000 7fa47ce48640 1 --2- 172.21.15.5:0/4237772485 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa47814dd70 0x7fa47814e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:17.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.608+0000 7fa47ce48640 1 -- 172.21.15.5:0/4237772485 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa47813bea0 msgr2=0x7fa47813c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:17.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.608+0000 7fa47ce48640 1 --2- 172.21.15.5:0/4237772485 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa47813bea0 0x7fa47813c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:17.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.608+0000 7fa47ce48640 1 -- 172.21.15.5:0/4237772485 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa47814e710 con 0x7fa460004e70 2024-03-18T15:34:17.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.608+0000 7fa47d649640 1 --2- 172.21.15.5:0/4237772485 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa47813bea0 0x7fa47813c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:17.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.612+0000 7fa47ce48640 1 --2- 172.21.15.5:0/4237772485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa460004e70 0x7fa478149ff0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa468011a00 tx=0x7fa468011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:17.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.612+0000 7fa4757fa640 1 -- 172.21.15.5:0/4237772485 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46800fb50 con 0x7fa460004e70 2024-03-18T15:34:17.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.612+0000 7fa4757fa640 1 -- 172.21.15.5:0/4237772485 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa46800fcf0 con 0x7fa460004e70 2024-03-18T15:34:17.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.612+0000 7fa47f0ac640 1 -- 172.21.15.5:0/4237772485 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa478144820 con 0x7fa460004e70 2024-03-18T15:34:17.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.612+0000 7fa4757fa640 1 -- 172.21.15.5:0/4237772485 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46800fea0 con 0x7fa460004e70 2024-03-18T15:34:17.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.612+0000 7fa47f0ac640 1 -- 172.21.15.5:0/4237772485 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa47813c860 con 0x7fa460004e70 2024-03-18T15:34:17.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.612+0000 7fa47f0ac640 1 -- 172.21.15.5:0/4237772485 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa47804f550 con 0x7fa460004e70 2024-03-18T15:34:17.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.612+0000 7fa4757fa640 1 -- 172.21.15.5:0/4237772485 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fa468012040 con 0x7fa460004e70 2024-03-18T15:34:17.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.612+0000 7fa4757fa640 1 --2- 172.21.15.5:0/4237772485 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa464067690 0x7fa464069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:17.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.612+0000 7fa4757fa640 1 -- 172.21.15.5:0/4237772485 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 4857+0+0 (secure 0 0 0) 0x7fa468085b90 con 0x7fa460004e70 2024-03-18T15:34:17.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.612+0000 7fa477fff640 1 --2- 172.21.15.5:0/4237772485 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa464067690 0x7fa464069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:17.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.612+0000 7fa477fff640 1 --2- 172.21.15.5:0/4237772485 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa464067690 0x7fa464069b50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa46c001040 tx=0x7fa46c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:17.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.616+0000 7fa4757fa640 1 -- 172.21.15.5:0/4237772485 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fa468019030 con 0x7fa460004e70 2024-03-18T15:34:17.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.836+0000 7fa47f0ac640 1 -- 172.21.15.5:0/4237772485 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7fa4780b7140 con 0x7fa460004e70 2024-03-18T15:34:17.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.840+0000 7fa4757fa640 1 -- 172.21.15.5:0/4237772485 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7fa468056c10 con 0x7fa460004e70 2024-03-18T15:34:17.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.840+0000 7fa47f0ac640 1 -- 172.21.15.5:0/4237772485 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa464067690 msgr2=0x7fa464069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:17.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.840+0000 7fa47f0ac640 1 --2- 172.21.15.5:0/4237772485 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa464067690 0x7fa464069b50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa46c001040 tx=0x7fa46c009040 comp rx=0 tx=0).stop 2024-03-18T15:34:17.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.840+0000 7fa47f0ac640 1 -- 172.21.15.5:0/4237772485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa460004e70 msgr2=0x7fa478149ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:17.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.840+0000 7fa47f0ac640 1 --2- 172.21.15.5:0/4237772485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa460004e70 0x7fa478149ff0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa468011a00 tx=0x7fa468011ec0 comp rx=0 tx=0).stop 2024-03-18T15:34:17.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.840+0000 7fa47f0ac640 1 -- 172.21.15.5:0/4237772485 shutdown_connections 2024-03-18T15:34:17.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.840+0000 7fa47f0ac640 1 --2- 172.21.15.5:0/4237772485 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa464067690 0x7fa464069b50 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:17.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.840+0000 7fa47f0ac640 1 --2- 172.21.15.5:0/4237772485 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa47813bea0 0x7fa47813c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:17.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.840+0000 7fa47f0ac640 1 --2- 172.21.15.5:0/4237772485 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa47814dd70 0x7fa47814e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:17.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.840+0000 7fa47f0ac640 1 --2- 172.21.15.5:0/4237772485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa460004e70 0x7fa478149ff0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:17.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.840+0000 7fa47f0ac640 1 -- 172.21.15.5:0/4237772485 >> 172.21.15.5:0/4237772485 conn(0x7fa4780673c0 msgr2=0x7fa4780b7de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:17.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.840+0000 7fa47f0ac640 1 -- 172.21.15.5:0/4237772485 shutdown_connections 2024-03-18T15:34:17.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.840+0000 7fa47f0ac640 1 -- 172.21.15.5:0/4237772485 wait complete. 2024-03-18T15:34:17.861 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-03-18T15:34:17.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.980+0000 7f52a7aea640 1 Processor -- start 2024-03-18T15:34:17.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.980+0000 7f52a7aea640 1 -- start start 2024-03-18T15:34:17.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.984+0000 7f52a7aea640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f52a006d2d0 0x7f52a00b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:17.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.984+0000 7f52a7aea640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f52a013f260 0x7f52a013f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:17.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.984+0000 7f52a7aea640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52a00b0ea0 con 0x7f52a01453a0 2024-03-18T15:34:17.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.984+0000 7f52a7aea640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f52a00728c0 con 0x7f52a013f260 2024-03-18T15:34:17.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.984+0000 7f52a7aea640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f52a006d6b0 con 0x7f52a006d2d0 2024-03-18T15:34:17.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.984+0000 7f52a5085640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f52a013f260 0x7f52a013f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:17.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.984+0000 7f52a5886640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f52a006d2d0 0x7f52a00b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:17.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.984+0000 7f52a6087640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f52a01453a0 0x7f52a0145810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.5:38118/0 (socket says 172.21.15.5:38118) 2024-03-18T15:34:17.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.984+0000 7f52a6087640 1 -- 172.21.15.5:0/2802982279 learned_addr learned my addr 172.21.15.5:0/2802982279 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:17.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.984+0000 7f52a5085640 1 -- 172.21.15.5:0/2802982279 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f52a006d2d0 msgr2=0x7f52a00b23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:17.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.984+0000 7f52a5085640 1 --2- 172.21.15.5:0/2802982279 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f52a006d2d0 0x7f52a00b23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:17.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.984+0000 7f52a5085640 1 -- 172.21.15.5:0/2802982279 >> v1:172.21.15.80:6789/0 conn(0x7f52a01453a0 legacy=0x7f52a0145810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:17.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.984+0000 7f52a5085640 1 -- 172.21.15.5:0/2802982279 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52a00b0bc0 con 0x7f52a013f260 2024-03-18T15:34:17.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.984+0000 7f52a5085640 1 --2- 172.21.15.5:0/2802982279 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f52a013f260 0x7f52a013f6a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5294009d90 tx=0x7f529401c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec81e6435664debf server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:17.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.988+0000 7f52a4884640 1 -- 172.21.15.5:0/2802982279 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52940b2070 con 0x7f52a013f260 2024-03-18T15:34:17.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.988+0000 7f52a4884640 1 -- 172.21.15.5:0/2802982279 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52940a58b0 con 0x7f52a013f260 2024-03-18T15:34:17.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.988+0000 7f52a4884640 1 -- 172.21.15.5:0/2802982279 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52940a5c80 con 0x7f52a013f260 2024-03-18T15:34:17.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.988+0000 7f52a7aea640 1 -- 172.21.15.5:0/2802982279 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f52a013f260 msgr2=0x7f52a013f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:17.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.988+0000 7f52a7aea640 1 --2- 172.21.15.5:0/2802982279 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f52a013f260 0x7f52a013f6a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5294009d90 tx=0x7f529401c8e0 comp rx=0 tx=0).stop 2024-03-18T15:34:17.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.988+0000 7f52a7aea640 1 -- 172.21.15.5:0/2802982279 shutdown_connections 2024-03-18T15:34:17.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.988+0000 7f52a7aea640 1 --2- 172.21.15.5:0/2802982279 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f52a013f260 0x7f52a013f6a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:17.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.988+0000 7f52a7aea640 1 --2- 172.21.15.5:0/2802982279 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f52a006d2d0 0x7f52a00b23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:17.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.988+0000 7f52a7aea640 1 -- 172.21.15.5:0/2802982279 >> 172.21.15.5:0/2802982279 conn(0x7f52a00b5670 msgr2=0x7f52a00b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:17.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.988+0000 7f52a7aea640 1 -- 172.21.15.5:0/2802982279 shutdown_connections 2024-03-18T15:34:17.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.988+0000 7f52a7aea640 1 -- 172.21.15.5:0/2802982279 wait complete. 2024-03-18T15:34:17.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.988+0000 7f52a7aea640 1 Processor -- start 2024-03-18T15:34:17.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.988+0000 7f52a7aea640 1 -- start start 2024-03-18T15:34:17.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.988+0000 7f52a7aea640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f52a006d2d0 0x7f52a0140eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:17.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.988+0000 7f52a7aea640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f52a013f260 0x7f52a01392a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:17.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.988+0000 7f52a7aea640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52a01453a0 0x7f52a0136020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:17.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.988+0000 7f52a7aea640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f52a00b52c0 con 0x7f52a01453a0 2024-03-18T15:34:17.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.988+0000 7f52a7aea640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f52a00b5140 con 0x7f52a013f260 2024-03-18T15:34:17.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.988+0000 7f52a7aea640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f52a0072d30 con 0x7f52a006d2d0 2024-03-18T15:34:17.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.988+0000 7f52a6087640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52a01453a0 0x7f52a0136020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:17.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.988+0000 7f52a6087640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52a01453a0 0x7f52a0136020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45950/0 (socket says 172.21.15.5:45950) 2024-03-18T15:34:17.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.988+0000 7f52a6087640 1 -- 172.21.15.5:0/3166916258 learned_addr learned my addr 172.21.15.5:0/3166916258 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:17.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.988+0000 7f52a5886640 1 --2- 172.21.15.5:0/3166916258 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f52a006d2d0 0x7f52a0140eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:17.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.988+0000 7f52a5085640 1 --2- 172.21.15.5:0/3166916258 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f52a013f260 0x7f52a01392a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:17.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.988+0000 7f52a6087640 1 -- 172.21.15.5:0/3166916258 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f52a006d2d0 msgr2=0x7f52a0140eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:17.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.988+0000 7f52a6087640 1 --2- 172.21.15.5:0/3166916258 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f52a006d2d0 0x7f52a0140eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:17.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.988+0000 7f52a6087640 1 -- 172.21.15.5:0/3166916258 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f52a013f260 msgr2=0x7f52a01392a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:17.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.988+0000 7f52a6087640 1 --2- 172.21.15.5:0/3166916258 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f52a013f260 0x7f52a01392a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:17.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.988+0000 7f52a6087640 1 -- 172.21.15.5:0/3166916258 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52a0072660 con 0x7f52a01453a0 2024-03-18T15:34:17.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.988+0000 7f52a6087640 1 --2- 172.21.15.5:0/3166916258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52a01453a0 0x7f52a0136020 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f529800c3c0 tx=0x7f529800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:17.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.988+0000 7f529e7fc640 1 -- 172.21.15.5:0/3166916258 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5298017070 con 0x7f52a01453a0 2024-03-18T15:34:17.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.988+0000 7f529e7fc640 1 -- 172.21.15.5:0/3166916258 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5298005cf0 con 0x7f52a01453a0 2024-03-18T15:34:17.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.988+0000 7f52a7aea640 1 -- 172.21.15.5:0/3166916258 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52a0147150 con 0x7f52a01453a0 2024-03-18T15:34:17.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.988+0000 7f529e7fc640 1 -- 172.21.15.5:0/3166916258 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5298007310 con 0x7f52a01453a0 2024-03-18T15:34:17.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.988+0000 7f52a7aea640 1 -- 172.21.15.5:0/3166916258 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52a01473c0 con 0x7f52a01453a0 2024-03-18T15:34:17.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.992+0000 7f52a7aea640 1 -- 172.21.15.5:0/3166916258 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52a004f550 con 0x7f52a01453a0 2024-03-18T15:34:17.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.992+0000 7f529e7fc640 1 -- 172.21.15.5:0/3166916258 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f529801a070 con 0x7f52a01453a0 2024-03-18T15:34:17.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.992+0000 7f529e7fc640 1 --2- 172.21.15.5:0/3166916258 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f528c067660 0x7f528c069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:17.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.992+0000 7f529e7fc640 1 -- 172.21.15.5:0/3166916258 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 4857+0+0 (secure 0 0 0) 0x7f5298084870 con 0x7f52a01453a0 2024-03-18T15:34:17.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.992+0000 7f52a5886640 1 --2- 172.21.15.5:0/3166916258 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f528c067660 0x7f528c069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:17.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.992+0000 7f52a5886640 1 --2- 172.21.15.5:0/3166916258 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f528c067660 0x7f528c069b20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5290009a50 tx=0x7f5290008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:17.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:17.996+0000 7f529e7fc640 1 -- 172.21.15.5:0/3166916258 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f5298055850 con 0x7f52a01453a0 2024-03-18T15:34:18.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.220+0000 7f52a7aea640 1 -- 172.21.15.5:0/3166916258 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f52a01375e0 con 0x7f52a01453a0 2024-03-18T15:34:18.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.224+0000 7f529e7fc640 1 -- 172.21.15.5:0/3166916258 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7f5298055150 con 0x7f52a01453a0 2024-03-18T15:34:18.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.224+0000 7f52a7aea640 1 -- 172.21.15.5:0/3166916258 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f528c067660 msgr2=0x7f528c069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:18.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.224+0000 7f52a7aea640 1 --2- 172.21.15.5:0/3166916258 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f528c067660 0x7f528c069b20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5290009a50 tx=0x7f5290008040 comp rx=0 tx=0).stop 2024-03-18T15:34:18.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.224+0000 7f52a7aea640 1 -- 172.21.15.5:0/3166916258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52a01453a0 msgr2=0x7f52a0136020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:18.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.224+0000 7f52a7aea640 1 --2- 172.21.15.5:0/3166916258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52a01453a0 0x7f52a0136020 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f529800c3c0 tx=0x7f529800c880 comp rx=0 tx=0).stop 2024-03-18T15:34:18.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.224+0000 7f52a7aea640 1 -- 172.21.15.5:0/3166916258 shutdown_connections 2024-03-18T15:34:18.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.224+0000 7f52a7aea640 1 --2- 172.21.15.5:0/3166916258 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f528c067660 0x7f528c069b20 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:18.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.224+0000 7f52a7aea640 1 --2- 172.21.15.5:0/3166916258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52a01453a0 0x7f52a0136020 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:18.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.224+0000 7f52a7aea640 1 --2- 172.21.15.5:0/3166916258 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f52a013f260 0x7f52a01392a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:18.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.224+0000 7f52a7aea640 1 --2- 172.21.15.5:0/3166916258 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f52a006d2d0 0x7f52a0140eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:18.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.224+0000 7f52a7aea640 1 -- 172.21.15.5:0/3166916258 >> 172.21.15.5:0/3166916258 conn(0x7f52a00b5670 msgr2=0x7f52a00b22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:18.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.224+0000 7f52a7aea640 1 -- 172.21.15.5:0/3166916258 shutdown_connections 2024-03-18T15:34:18.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.224+0000 7f52a7aea640 1 -- 172.21.15.5:0/3166916258 wait complete. 2024-03-18T15:34:18.245 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-03-18T15:34:18.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.360+0000 7fa6f8228640 1 Processor -- start 2024-03-18T15:34:18.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.360+0000 7fa6f8228640 1 -- start start 2024-03-18T15:34:18.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.360+0000 7fa6f8228640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa6f00b5710 0x7fa6f014a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:18.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.360+0000 7fa6f8228640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa6f014ae60 0x7fa6f0140ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:18.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.360+0000 7fa6f8228640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6f00b9d50 con 0x7fa6f0141430 2024-03-18T15:34:18.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.360+0000 7fa6f8228640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fa6f00b4320 con 0x7fa6f014ae60 2024-03-18T15:34:18.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.360+0000 7fa6f8228640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fa6f00b4570 con 0x7fa6f00b5710 2024-03-18T15:34:18.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.360+0000 7fa6f57c3640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa6f014ae60 0x7fa6f0140ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:18.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.360+0000 7fa6f57c3640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fa6f014ae60 0x7fa6f0140ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45954/0 (socket says 172.21.15.5:45954) 2024-03-18T15:34:18.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.360+0000 7fa6f57c3640 1 -- 172.21.15.5:0/2885275792 learned_addr learned my addr 172.21.15.5:0/2885275792 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:18.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.360+0000 7fa6f5fc4640 1 --2- 172.21.15.5:0/2885275792 >> v2:172.21.15.80:3300/0 conn(0x7fa6f00b5710 0x7fa6f014a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:18.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.360+0000 7fa6f57c3640 1 -- 172.21.15.5:0/2885275792 >> v2:172.21.15.80:3300/0 conn(0x7fa6f00b5710 msgr2=0x7fa6f014a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:18.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.360+0000 7fa6f57c3640 1 --2- 172.21.15.5:0/2885275792 >> v2:172.21.15.80:3300/0 conn(0x7fa6f00b5710 0x7fa6f014a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:18.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.360+0000 7fa6f57c3640 1 -- 172.21.15.5:0/2885275792 >> v1:172.21.15.80:6789/0 conn(0x7fa6f0141430 legacy=0x7fa6f00b7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:18.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.360+0000 7fa6f57c3640 1 -- 172.21.15.5:0/2885275792 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f00b4c30 con 0x7fa6f014ae60 2024-03-18T15:34:18.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.364+0000 7fa6f57c3640 1 --2- 172.21.15.5:0/2885275792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6f014ae60 0x7fa6f0140ef0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa6f006c3e0 tx=0x7fa6e001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18348f5de48ee5a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:18.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.364+0000 7fa6f4fc2640 1 -- 172.21.15.5:0/2885275792 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e00b0070 con 0x7fa6f014ae60 2024-03-18T15:34:18.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.364+0000 7fa6f4fc2640 1 -- 172.21.15.5:0/2885275792 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6e00ab600 con 0x7fa6f014ae60 2024-03-18T15:34:18.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.364+0000 7fa6f4fc2640 1 -- 172.21.15.5:0/2885275792 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e00ab9d0 con 0x7fa6f014ae60 2024-03-18T15:34:18.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.364+0000 7fa6f8228640 1 -- 172.21.15.5:0/2885275792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6f014ae60 msgr2=0x7fa6f0140ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:18.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.364+0000 7fa6f8228640 1 --2- 172.21.15.5:0/2885275792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6f014ae60 0x7fa6f0140ef0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa6f006c3e0 tx=0x7fa6e001c3c0 comp rx=0 tx=0).stop 2024-03-18T15:34:18.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.364+0000 7fa6f8228640 1 -- 172.21.15.5:0/2885275792 shutdown_connections 2024-03-18T15:34:18.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.364+0000 7fa6f8228640 1 --2- 172.21.15.5:0/2885275792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6f014ae60 0x7fa6f0140ef0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:18.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.364+0000 7fa6f8228640 1 --2- 172.21.15.5:0/2885275792 >> v2:172.21.15.80:3300/0 conn(0x7fa6f00b5710 0x7fa6f014a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:18.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.364+0000 7fa6f8228640 1 -- 172.21.15.5:0/2885275792 >> 172.21.15.5:0/2885275792 conn(0x7fa6f00673c0 msgr2=0x7fa6f00677d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:18.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.364+0000 7fa6f8228640 1 -- 172.21.15.5:0/2885275792 shutdown_connections 2024-03-18T15:34:18.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.364+0000 7fa6f8228640 1 -- 172.21.15.5:0/2885275792 wait complete. 2024-03-18T15:34:18.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.364+0000 7fa6f8228640 1 Processor -- start 2024-03-18T15:34:18.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.364+0000 7fa6f8228640 1 -- start start 2024-03-18T15:34:18.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.368+0000 7fa6f8228640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6f00b5710 0x7fa6f0136610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:18.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.368+0000 7fa6f8228640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa6f0141430 0x7fa6f013b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:18.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.368+0000 7fa6f8228640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6f014ae60 0x7fa6f0130ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:18.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.368+0000 7fa6f8228640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f0141880 con 0x7fa6f00b5710 2024-03-18T15:34:18.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.368+0000 7fa6f8228640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f014b2c0 con 0x7fa6f014ae60 2024-03-18T15:34:18.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.368+0000 7fa6f8228640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fa6f0141a00 con 0x7fa6f0141430 2024-03-18T15:34:18.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.368+0000 7fa6f5fc4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6f00b5710 0x7fa6f0136610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:18.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.368+0000 7fa6f5fc4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6f00b5710 0x7fa6f0136610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45964/0 (socket says 172.21.15.5:45964) 2024-03-18T15:34:18.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.368+0000 7fa6f5fc4640 1 -- 172.21.15.5:0/2902909942 learned_addr learned my addr 172.21.15.5:0/2902909942 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:18.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.368+0000 7fa6f5fc4640 1 -- 172.21.15.5:0/2902909942 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa6f0141430 msgr2=0x7fa6f013b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:18.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.368+0000 7fa6f5fc4640 1 --2- 172.21.15.5:0/2902909942 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa6f0141430 0x7fa6f013b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:18.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.368+0000 7fa6f5fc4640 1 -- 172.21.15.5:0/2902909942 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6f014ae60 msgr2=0x7fa6f0130ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:18.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.368+0000 7fa6f5fc4640 1 --2- 172.21.15.5:0/2902909942 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6f014ae60 0x7fa6f0130ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:18.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.368+0000 7fa6f5fc4640 1 -- 172.21.15.5:0/2902909942 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f013b910 con 0x7fa6f00b5710 2024-03-18T15:34:18.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.368+0000 7fa6f5fc4640 1 --2- 172.21.15.5:0/2902909942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6f00b5710 0x7fa6f0136610 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec00d830 tx=0x7fa6ec00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:18.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.368+0000 7fa6e67fc640 1 -- 172.21.15.5:0/2902909942 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec00b6c0 con 0x7fa6f00b5710 2024-03-18T15:34:18.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.368+0000 7fa6e67fc640 1 -- 172.21.15.5:0/2902909942 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6ec019070 con 0x7fa6f00b5710 2024-03-18T15:34:18.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.368+0000 7fa6f8228640 1 -- 172.21.15.5:0/2902909942 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6f013bbf0 con 0x7fa6f00b5710 2024-03-18T15:34:18.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.368+0000 7fa6e67fc640 1 -- 172.21.15.5:0/2902909942 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec00bae0 con 0x7fa6f00b5710 2024-03-18T15:34:18.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.368+0000 7fa6f8228640 1 -- 172.21.15.5:0/2902909942 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6f013a3b0 con 0x7fa6f00b5710 2024-03-18T15:34:18.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.368+0000 7fa6f8228640 1 -- 172.21.15.5:0/2902909942 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6f004f550 con 0x7fa6f00b5710 2024-03-18T15:34:18.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.368+0000 7fa6e67fc640 1 -- 172.21.15.5:0/2902909942 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fa6ec01e460 con 0x7fa6f00b5710 2024-03-18T15:34:18.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.368+0000 7fa6e67fc640 1 --2- 172.21.15.5:0/2902909942 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa6d4067690 0x7fa6d4069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:18.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.368+0000 7fa6e67fc640 1 -- 172.21.15.5:0/2902909942 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 4857+0+0 (secure 0 0 0) 0x7fa6ec085bd0 con 0x7fa6f00b5710 2024-03-18T15:34:18.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.368+0000 7fa6f57c3640 1 --2- 172.21.15.5:0/2902909942 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa6d4067690 0x7fa6d4069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:18.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.368+0000 7fa6f57c3640 1 --2- 172.21.15.5:0/2902909942 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa6d4067690 0x7fa6d4069b50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa6e00ad000 tx=0x7fa6e0004740 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:18.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.372+0000 7fa6e67fc640 1 -- 172.21.15.5:0/2902909942 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fa6ec056c30 con 0x7fa6f00b5710 2024-03-18T15:34:18.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.592+0000 7fa6f8228640 1 -- 172.21.15.5:0/2902909942 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7fa6f00b7140 con 0x7fa6f00b5710 2024-03-18T15:34:18.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.596+0000 7fa6e67fc640 1 -- 172.21.15.5:0/2902909942 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7fa6ec056530 con 0x7fa6f00b5710 2024-03-18T15:34:18.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.596+0000 7fa6f8228640 1 -- 172.21.15.5:0/2902909942 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa6d4067690 msgr2=0x7fa6d4069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:18.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.596+0000 7fa6f8228640 1 --2- 172.21.15.5:0/2902909942 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa6d4067690 0x7fa6d4069b50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa6e00ad000 tx=0x7fa6e0004740 comp rx=0 tx=0).stop 2024-03-18T15:34:18.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.596+0000 7fa6f8228640 1 -- 172.21.15.5:0/2902909942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6f00b5710 msgr2=0x7fa6f0136610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:18.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.596+0000 7fa6f8228640 1 --2- 172.21.15.5:0/2902909942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6f00b5710 0x7fa6f0136610 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec00d830 tx=0x7fa6ec00dcf0 comp rx=0 tx=0).stop 2024-03-18T15:34:18.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.600+0000 7fa6f8228640 1 -- 172.21.15.5:0/2902909942 shutdown_connections 2024-03-18T15:34:18.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.600+0000 7fa6f8228640 1 --2- 172.21.15.5:0/2902909942 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa6d4067690 0x7fa6d4069b50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:18.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.600+0000 7fa6f8228640 1 --2- 172.21.15.5:0/2902909942 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6f014ae60 0x7fa6f0130ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:18.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.600+0000 7fa6f8228640 1 --2- 172.21.15.5:0/2902909942 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa6f0141430 0x7fa6f013b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:18.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.600+0000 7fa6f8228640 1 --2- 172.21.15.5:0/2902909942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6f00b5710 0x7fa6f0136610 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:18.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.600+0000 7fa6f8228640 1 -- 172.21.15.5:0/2902909942 >> 172.21.15.5:0/2902909942 conn(0x7fa6f00673c0 msgr2=0x7fa6f013ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:18.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.600+0000 7fa6f8228640 1 -- 172.21.15.5:0/2902909942 shutdown_connections 2024-03-18T15:34:18.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.600+0000 7fa6f8228640 1 -- 172.21.15.5:0/2902909942 wait complete. 2024-03-18T15:34:18.619 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-03-18T15:34:18.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.744+0000 7fc61bdbb640 1 Processor -- start 2024-03-18T15:34:18.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.744+0000 7fc61bdbb640 1 -- start start 2024-03-18T15:34:18.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.744+0000 7fc61bdbb640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc61413f260 0x7fc61413f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:18.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.744+0000 7fc61bdbb640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6140b2ab0 con 0x7fc61414dc70 2024-03-18T15:34:18.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.744+0000 7fc61bdbb640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6140b0ea0 con 0x7fc61406d2d0 2024-03-18T15:34:18.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.744+0000 7fc61bdbb640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fc6140728c0 con 0x7fc61413f260 2024-03-18T15:34:18.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.744+0000 7fc619356640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fc61413f260 0x7fc61413f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:18.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.744+0000 7fc61a358640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fc61414dc70 0x7fc614150060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51568/0 (socket says 172.21.15.5:51568) 2024-03-18T15:34:18.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.744+0000 7fc61a358640 1 -- 172.21.15.5:0/4079467506 learned_addr learned my addr 172.21.15.5:0/4079467506 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:18.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.744+0000 7fc619356640 1 -- 172.21.15.5:0/4079467506 >> v1:172.21.15.80:6789/0 conn(0x7fc61406d2d0 legacy=0x7fc6140b23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:18.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.744+0000 7fc619356640 1 -- 172.21.15.5:0/4079467506 >> v1:172.21.15.5:6789/0 conn(0x7fc61414dc70 legacy=0x7fc614150060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:18.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.744+0000 7fc618b55640 1 -- 172.21.15.5:0/4079467506 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3394527888 0 0) 0x7fc6140b2ab0 con 0x7fc61414dc70 2024-03-18T15:34:18.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.744+0000 7fc619356640 1 -- 172.21.15.5:0/4079467506 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc61406d6b0 con 0x7fc61413f260 2024-03-18T15:34:18.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.748+0000 7fc619356640 1 --2- 172.21.15.5:0/4079467506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc61413f260 0x7fc61413f6a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc6040097c0 tx=0x7fc60401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86b116f20df0cda8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:18.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.748+0000 7fc618b55640 1 -- 172.21.15.5:0/4079467506 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6040b0070 con 0x7fc61413f260 2024-03-18T15:34:18.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.748+0000 7fc618b55640 1 -- 172.21.15.5:0/4079467506 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6040ab600 con 0x7fc61413f260 2024-03-18T15:34:18.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.748+0000 7fc618b55640 1 -- 172.21.15.5:0/4079467506 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6040ab9d0 con 0x7fc61413f260 2024-03-18T15:34:18.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.748+0000 7fc61bdbb640 1 -- 172.21.15.5:0/4079467506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc61413f260 msgr2=0x7fc61413f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:18.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.748+0000 7fc61bdbb640 1 --2- 172.21.15.5:0/4079467506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc61413f260 0x7fc61413f6a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc6040097c0 tx=0x7fc60401c3d0 comp rx=0 tx=0).stop 2024-03-18T15:34:18.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.748+0000 7fc61bdbb640 1 -- 172.21.15.5:0/4079467506 shutdown_connections 2024-03-18T15:34:18.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.748+0000 7fc61bdbb640 1 --2- 172.21.15.5:0/4079467506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc61413f260 0x7fc61413f6a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:18.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.748+0000 7fc61bdbb640 1 -- 172.21.15.5:0/4079467506 >> 172.21.15.5:0/4079467506 conn(0x7fc6140b5670 msgr2=0x7fc6140b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:18.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.748+0000 7fc61bdbb640 1 -- 172.21.15.5:0/4079467506 shutdown_connections 2024-03-18T15:34:18.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.748+0000 7fc61bdbb640 1 -- 172.21.15.5:0/4079467506 wait complete. 2024-03-18T15:34:18.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.748+0000 7fc61bdbb640 1 Processor -- start 2024-03-18T15:34:18.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.748+0000 7fc61bdbb640 1 -- start start 2024-03-18T15:34:18.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.748+0000 7fc61bdbb640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc61406d2d0 0x7fc614145690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:18.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.752+0000 7fc61bdbb640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc61413f260 0x7fc6141de250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:18.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.752+0000 7fc61bdbb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc61414dc70 0x7fc6141de790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:18.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.752+0000 7fc61bdbb640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc6140b0960 con 0x7fc61414dc70 2024-03-18T15:34:18.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.752+0000 7fc61bdbb640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc6140b07e0 con 0x7fc61406d2d0 2024-03-18T15:34:18.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.752+0000 7fc61bdbb640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fc614072d50 con 0x7fc61413f260 2024-03-18T15:34:18.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.752+0000 7fc61a358640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc61414dc70 0x7fc6141de790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:18.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.752+0000 7fc61a358640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc61414dc70 0x7fc6141de790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45982/0 (socket says 172.21.15.5:45982) 2024-03-18T15:34:18.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.752+0000 7fc61a358640 1 -- 172.21.15.5:0/2195144826 learned_addr learned my addr 172.21.15.5:0/2195144826 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:18.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.752+0000 7fc619356640 1 --2- 172.21.15.5:0/2195144826 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc61413f260 0x7fc6141de250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:18.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.752+0000 7fc619b57640 1 --2- 172.21.15.5:0/2195144826 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc61406d2d0 0x7fc614145690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:18.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.752+0000 7fc61a358640 1 -- 172.21.15.5:0/2195144826 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc61413f260 msgr2=0x7fc6141de250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:18.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.752+0000 7fc61a358640 1 --2- 172.21.15.5:0/2195144826 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc61413f260 0x7fc6141de250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:18.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.752+0000 7fc61a358640 1 -- 172.21.15.5:0/2195144826 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc61406d2d0 msgr2=0x7fc614145690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:18.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.752+0000 7fc61a358640 1 --2- 172.21.15.5:0/2195144826 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc61406d2d0 0x7fc614145690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:18.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.752+0000 7fc61a358640 1 -- 172.21.15.5:0/2195144826 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc614141250 con 0x7fc61414dc70 2024-03-18T15:34:18.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.752+0000 7fc61a358640 1 --2- 172.21.15.5:0/2195144826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc61414dc70 0x7fc6141de790 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc61000c450 tx=0x7fc61000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:18.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.752+0000 7fc60a7fc640 1 -- 172.21.15.5:0/2195144826 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6100074e0 con 0x7fc61414dc70 2024-03-18T15:34:18.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.752+0000 7fc60a7fc640 1 -- 172.21.15.5:0/2195144826 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6100047f0 con 0x7fc61414dc70 2024-03-18T15:34:18.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.752+0000 7fc60a7fc640 1 -- 172.21.15.5:0/2195144826 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc610007680 con 0x7fc61414dc70 2024-03-18T15:34:18.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.752+0000 7fc61bdbb640 1 -- 172.21.15.5:0/2195144826 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc61413feb0 con 0x7fc61414dc70 2024-03-18T15:34:18.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.752+0000 7fc61bdbb640 1 -- 172.21.15.5:0/2195144826 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc614140190 con 0x7fc61414dc70 2024-03-18T15:34:18.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.752+0000 7fc61bdbb640 1 -- 172.21.15.5:0/2195144826 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc61404f550 con 0x7fc61414dc70 2024-03-18T15:34:18.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.752+0000 7fc60a7fc640 1 -- 172.21.15.5:0/2195144826 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fc610011070 con 0x7fc61414dc70 2024-03-18T15:34:18.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.752+0000 7fc60a7fc640 1 --2- 172.21.15.5:0/2195144826 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fc5f8067660 0x7fc5f8069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:18.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.752+0000 7fc60a7fc640 1 -- 172.21.15.5:0/2195144826 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 4857+0+0 (secure 0 0 0) 0x7fc610085150 con 0x7fc61414dc70 2024-03-18T15:34:18.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.752+0000 7fc619b57640 1 --2- 172.21.15.5:0/2195144826 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fc5f8067660 0x7fc5f8069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:18.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.752+0000 7fc619b57640 1 --2- 172.21.15.5:0/2195144826 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fc5f8067660 0x7fc5f8069b20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc60c009f20 tx=0x7fc60c0094d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:18.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.756+0000 7fc60a7fc640 1 -- 172.21.15.5:0/2195144826 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fc610056110 con 0x7fc61414dc70 2024-03-18T15:34:18.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.968+0000 7fc61bdbb640 1 -- 172.21.15.5:0/2195144826 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7fc6140b4b60 con 0x7fc61414dc70 2024-03-18T15:34:18.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.972+0000 7fc60a7fc640 1 -- 172.21.15.5:0/2195144826 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) ==== 97+0+0 (secure 0 0 0) 0x7fc610055a10 con 0x7fc61414dc70 2024-03-18T15:34:18.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.976+0000 7fc61bdbb640 1 -- 172.21.15.5:0/2195144826 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fc5f8067660 msgr2=0x7fc5f8069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:18.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.976+0000 7fc61bdbb640 1 --2- 172.21.15.5:0/2195144826 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fc5f8067660 0x7fc5f8069b20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc60c009f20 tx=0x7fc60c0094d0 comp rx=0 tx=0).stop 2024-03-18T15:34:18.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.976+0000 7fc61bdbb640 1 -- 172.21.15.5:0/2195144826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc61414dc70 msgr2=0x7fc6141de790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:18.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.976+0000 7fc61bdbb640 1 --2- 172.21.15.5:0/2195144826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc61414dc70 0x7fc6141de790 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc61000c450 tx=0x7fc61000c910 comp rx=0 tx=0).stop 2024-03-18T15:34:18.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.976+0000 7fc61bdbb640 1 -- 172.21.15.5:0/2195144826 shutdown_connections 2024-03-18T15:34:18.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.976+0000 7fc61bdbb640 1 --2- 172.21.15.5:0/2195144826 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fc5f8067660 0x7fc5f8069b20 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:18.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.976+0000 7fc61bdbb640 1 --2- 172.21.15.5:0/2195144826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc61414dc70 0x7fc6141de790 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:18.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.976+0000 7fc61bdbb640 1 --2- 172.21.15.5:0/2195144826 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc61413f260 0x7fc6141de250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:18.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.976+0000 7fc61bdbb640 1 --2- 172.21.15.5:0/2195144826 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc61406d2d0 0x7fc614145690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:18.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.976+0000 7fc61bdbb640 1 -- 172.21.15.5:0/2195144826 >> 172.21.15.5:0/2195144826 conn(0x7fc6140b5670 msgr2=0x7fc61406c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:18.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.976+0000 7fc61bdbb640 1 -- 172.21.15.5:0/2195144826 shutdown_connections 2024-03-18T15:34:18.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:18.976+0000 7fc61bdbb640 1 -- 172.21.15.5:0/2195144826 wait complete. 2024-03-18T15:34:18.994 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-03-18T15:34:19.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.116+0000 7ff5e6dbe640 1 Processor -- start 2024-03-18T15:34:19.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.116+0000 7ff5e6dbe640 1 -- start start 2024-03-18T15:34:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.116+0000 7ff5e6dbe640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff5e014b030 0x7ff5e0140ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.116+0000 7ff5e6dbe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7ff5e0141430 0x7ff5e0151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.116+0000 7ff5e6dbe640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5e00b9d50 con 0x7ff5e00b5710 2024-03-18T15:34:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.116+0000 7ff5e6dbe640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7ff5e00b4320 con 0x7ff5e0141430 2024-03-18T15:34:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.116+0000 7ff5e6dbe640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7ff5e00b4570 con 0x7ff5e014b030 2024-03-18T15:34:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.116+0000 7ff5e4b5a640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff5e00b5710 0x7ff5e014a920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51584/0 (socket says 172.21.15.5:51584) 2024-03-18T15:34:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.116+0000 7ff5e4b5a640 1 -- 172.21.15.5:0/2784717010 learned_addr learned my addr 172.21.15.5:0/2784717010 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.116+0000 7ff5e535b640 1 --2- 172.21.15.5:0/2784717010 >> v2:172.21.15.5:3300/0 conn(0x7ff5e0141430 0x7ff5e0151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:19.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.116+0000 7ff5dffff640 1 --2- 172.21.15.5:0/2784717010 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff5e014b030 0x7ff5e0140ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.116+0000 7ff5df7fe640 1 -- 172.21.15.5:0/2784717010 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1104429449 0 0) 0x7ff5e00b9d50 con 0x7ff5e00b5710 2024-03-18T15:34:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.116+0000 7ff5df7fe640 1 -- 172.21.15.5:0/2784717010 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5cc003530 con 0x7ff5e00b5710 2024-03-18T15:34:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.116+0000 7ff5e535b640 1 -- 172.21.15.5:0/2784717010 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff5e014b030 msgr2=0x7ff5e0140ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.116+0000 7ff5e535b640 1 --2- 172.21.15.5:0/2784717010 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff5e014b030 0x7ff5e0140ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.116+0000 7ff5e535b640 1 -- 172.21.15.5:0/2784717010 >> v1:172.21.15.5:6789/0 conn(0x7ff5e00b5710 legacy=0x7ff5e014a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:19.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.116+0000 7ff5e535b640 1 -- 172.21.15.5:0/2784717010 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e00b4c30 con 0x7ff5e0141430 2024-03-18T15:34:19.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.120+0000 7ff5e535b640 1 --2- 172.21.15.5:0/2784717010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5e0141430 0x7ff5e0151cb0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff5d00097f0 tx=0x7ff5d001c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a94954a97587f055 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:19.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.120+0000 7ff5df7fe640 1 -- 172.21.15.5:0/2784717010 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d00b0070 con 0x7ff5e0141430 2024-03-18T15:34:19.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.120+0000 7ff5df7fe640 1 -- 172.21.15.5:0/2784717010 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5d00ab600 con 0x7ff5e0141430 2024-03-18T15:34:19.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.120+0000 7ff5df7fe640 1 -- 172.21.15.5:0/2784717010 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d00ab9d0 con 0x7ff5e0141430 2024-03-18T15:34:19.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.120+0000 7ff5e6dbe640 1 -- 172.21.15.5:0/2784717010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5e0141430 msgr2=0x7ff5e0151cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:19.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.120+0000 7ff5e6dbe640 1 --2- 172.21.15.5:0/2784717010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5e0141430 0x7ff5e0151cb0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff5d00097f0 tx=0x7ff5d001c380 comp rx=0 tx=0).stop 2024-03-18T15:34:19.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.120+0000 7ff5e6dbe640 1 -- 172.21.15.5:0/2784717010 shutdown_connections 2024-03-18T15:34:19.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.120+0000 7ff5e6dbe640 1 --2- 172.21.15.5:0/2784717010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5e0141430 0x7ff5e0151cb0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:19.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.120+0000 7ff5e6dbe640 1 --2- 172.21.15.5:0/2784717010 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff5e014b030 0x7ff5e0140ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:19.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.120+0000 7ff5e6dbe640 1 -- 172.21.15.5:0/2784717010 >> 172.21.15.5:0/2784717010 conn(0x7ff5e00673c0 msgr2=0x7ff5e00677d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:19.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.120+0000 7ff5e6dbe640 1 -- 172.21.15.5:0/2784717010 shutdown_connections 2024-03-18T15:34:19.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.120+0000 7ff5e6dbe640 1 -- 172.21.15.5:0/2784717010 wait complete. 2024-03-18T15:34:19.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.120+0000 7ff5e6dbe640 1 Processor -- start 2024-03-18T15:34:19.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.120+0000 7ff5e6dbe640 1 -- start start 2024-03-18T15:34:19.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.120+0000 7ff5e6dbe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff5e00b5710 0x7ff5e0149bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:19.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.120+0000 7ff5e6dbe640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff5e0141430 0x7ff5e00bbe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:19.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.120+0000 7ff5e6dbe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5e014b030 0x7ff5e0143fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:19.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.120+0000 7ff5e6dbe640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e00b5260 con 0x7ff5e014b030 2024-03-18T15:34:19.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.120+0000 7ff5e6dbe640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e00b50e0 con 0x7ff5e00b5710 2024-03-18T15:34:19.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.120+0000 7ff5e6dbe640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7ff5e0154a10 con 0x7ff5e0141430 2024-03-18T15:34:19.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.120+0000 7ff5e535b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5e014b030 0x7ff5e0143fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:19.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.120+0000 7ff5e535b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5e014b030 0x7ff5e0143fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46012/0 (socket says 172.21.15.5:46012) 2024-03-18T15:34:19.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.120+0000 7ff5e535b640 1 -- 172.21.15.5:0/636135336 learned_addr learned my addr 172.21.15.5:0/636135336 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:19.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.120+0000 7ff5e4b5a640 1 --2- 172.21.15.5:0/636135336 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff5e00b5710 0x7ff5e0149bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:19.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.120+0000 7ff5dffff640 1 --2- 172.21.15.5:0/636135336 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff5e0141430 0x7ff5e00bbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:19.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.120+0000 7ff5e535b640 1 -- 172.21.15.5:0/636135336 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff5e0141430 msgr2=0x7ff5e00bbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:19.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.120+0000 7ff5e535b640 1 --2- 172.21.15.5:0/636135336 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff5e0141430 0x7ff5e00bbe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:19.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.120+0000 7ff5e535b640 1 -- 172.21.15.5:0/636135336 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff5e00b5710 msgr2=0x7ff5e0149bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:19.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.120+0000 7ff5e535b640 1 --2- 172.21.15.5:0/636135336 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff5e00b5710 0x7ff5e0149bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:19.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.120+0000 7ff5e535b640 1 -- 172.21.15.5:0/636135336 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e00baaf0 con 0x7ff5e014b030 2024-03-18T15:34:19.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.120+0000 7ff5e4b5a640 1 --2- 172.21.15.5:0/636135336 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff5e00b5710 0x7ff5e0149bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:19.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.124+0000 7ff5e535b640 1 --2- 172.21.15.5:0/636135336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5e014b030 0x7ff5e0143fd0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff5d00ae040 tx=0x7ff5d0019120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:19.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.124+0000 7ff5dd7fa640 1 -- 172.21.15.5:0/636135336 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d00b0040 con 0x7ff5e014b030 2024-03-18T15:34:19.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.124+0000 7ff5dd7fa640 1 -- 172.21.15.5:0/636135336 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5d00b9070 con 0x7ff5e014b030 2024-03-18T15:34:19.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.124+0000 7ff5dd7fa640 1 -- 172.21.15.5:0/636135336 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5d00b4650 con 0x7ff5e014b030 2024-03-18T15:34:19.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.124+0000 7ff5e6dbe640 1 -- 172.21.15.5:0/636135336 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5e00bc360 con 0x7ff5e014b030 2024-03-18T15:34:19.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.124+0000 7ff5e6dbe640 1 -- 172.21.15.5:0/636135336 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5e00bc5e0 con 0x7ff5e014b030 2024-03-18T15:34:19.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.124+0000 7ff5e6dbe640 1 -- 172.21.15.5:0/636135336 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5e004f550 con 0x7ff5e014b030 2024-03-18T15:34:19.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.124+0000 7ff5dd7fa640 1 -- 172.21.15.5:0/636135336 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7ff5d00b47f0 con 0x7ff5e014b030 2024-03-18T15:34:19.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.124+0000 7ff5dd7fa640 1 --2- 172.21.15.5:0/636135336 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7ff5c0067660 0x7ff5c0069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:19.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.124+0000 7ff5dd7fa640 1 -- 172.21.15.5:0/636135336 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 4857+0+0 (secure 0 0 0) 0x7ff5d011f1d0 con 0x7ff5e014b030 2024-03-18T15:34:19.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.124+0000 7ff5e4b5a640 1 --2- 172.21.15.5:0/636135336 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7ff5c0067660 0x7ff5c0069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:19.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.124+0000 7ff5e4b5a640 1 --2- 172.21.15.5:0/636135336 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7ff5c0067660 0x7ff5c0069b20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8005c10 tx=0x7ff5d8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:19.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.128+0000 7ff5dd7fa640 1 -- 172.21.15.5:0/636135336 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7ff5e00bc360 con 0x7ff5e014b030 2024-03-18T15:34:19.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.336+0000 7ff5e6dbe640 1 -- 172.21.15.5:0/636135336 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7ff5e0148b80 con 0x7ff5e014b030 2024-03-18T15:34:19.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.340+0000 7ff5dd7fa640 1 -- 172.21.15.5:0/636135336 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7ff5e00baaf0 con 0x7ff5e014b030 2024-03-18T15:34:19.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.344+0000 7ff5e6dbe640 1 -- 172.21.15.5:0/636135336 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7ff5c0067660 msgr2=0x7ff5c0069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:19.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.344+0000 7ff5e6dbe640 1 --2- 172.21.15.5:0/636135336 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7ff5c0067660 0x7ff5c0069b20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8005c10 tx=0x7ff5d8009040 comp rx=0 tx=0).stop 2024-03-18T15:34:19.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.344+0000 7ff5e6dbe640 1 -- 172.21.15.5:0/636135336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5e014b030 msgr2=0x7ff5e0143fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:19.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.344+0000 7ff5e6dbe640 1 --2- 172.21.15.5:0/636135336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5e014b030 0x7ff5e0143fd0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff5d00ae040 tx=0x7ff5d0019120 comp rx=0 tx=0).stop 2024-03-18T15:34:19.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.344+0000 7ff5e6dbe640 1 -- 172.21.15.5:0/636135336 shutdown_connections 2024-03-18T15:34:19.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.344+0000 7ff5e6dbe640 1 --2- 172.21.15.5:0/636135336 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7ff5c0067660 0x7ff5c0069b20 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:19.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.344+0000 7ff5e6dbe640 1 --2- 172.21.15.5:0/636135336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5e014b030 0x7ff5e0143fd0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:19.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.344+0000 7ff5e6dbe640 1 --2- 172.21.15.5:0/636135336 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff5e0141430 0x7ff5e00bbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:19.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.344+0000 7ff5e6dbe640 1 --2- 172.21.15.5:0/636135336 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff5e00b5710 0x7ff5e0149bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:19.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.344+0000 7ff5e6dbe640 1 -- 172.21.15.5:0/636135336 >> 172.21.15.5:0/636135336 conn(0x7ff5e00673c0 msgr2=0x7ff5e00b7de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:19.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.344+0000 7ff5e6dbe640 1 -- 172.21.15.5:0/636135336 shutdown_connections 2024-03-18T15:34:19.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.344+0000 7ff5e6dbe640 1 -- 172.21.15.5:0/636135336 wait complete. 2024-03-18T15:34:19.362 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:34:19.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.484+0000 7f0a45a12640 1 Processor -- start 2024-03-18T15:34:19.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.484+0000 7f0a45a12640 1 -- start start 2024-03-18T15:34:19.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.484+0000 7f0a45a12640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0a4013f260 0x7f0a4013f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:19.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.484+0000 7f0a45a12640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a400b2ab0 con 0x7f0a4006d2d0 2024-03-18T15:34:19.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.484+0000 7f0a45a12640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a400b0ea0 con 0x7f0a4014dc70 2024-03-18T15:34:19.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.484+0000 7f0a45a12640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f0a400728c0 con 0x7f0a4013f260 2024-03-18T15:34:19.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.484+0000 7f0a3effd640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f0a4006d2d0 0x7f0a400b23a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:51596/0 (socket says 172.21.15.5:51596) 2024-03-18T15:34:19.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.484+0000 7f0a3e7fc640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0a4013f260 0x7f0a4013f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:19.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.484+0000 7f0a3effd640 1 -- 172.21.15.5:0/1209555332 learned_addr learned my addr 172.21.15.5:0/1209555332 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:19.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.484+0000 7f0a3dffb640 1 -- 172.21.15.5:0/1209555332 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1225515404 0 0) 0x7f0a400b2ab0 con 0x7f0a4006d2d0 2024-03-18T15:34:19.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.484+0000 7f0a3dffb640 1 -- 172.21.15.5:0/1209555332 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0a28003530 con 0x7f0a4006d2d0 2024-03-18T15:34:19.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.484+0000 7f0a3e7fc640 1 -- 172.21.15.5:0/1209555332 >> v1:172.21.15.80:6789/0 conn(0x7f0a4014dc70 legacy=0x7f0a40150060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:19.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.484+0000 7f0a3e7fc640 1 -- 172.21.15.5:0/1209555332 >> v1:172.21.15.5:6789/0 conn(0x7f0a4006d2d0 legacy=0x7f0a400b23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:19.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.484+0000 7f0a3e7fc640 1 -- 172.21.15.5:0/1209555332 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a4006d6b0 con 0x7f0a4013f260 2024-03-18T15:34:19.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.488+0000 7f0a3e7fc640 1 --2- 172.21.15.5:0/1209555332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a4013f260 0x7f0a4013f6a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0a340097c0 tx=0x7f0a3401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=317c541b6cfb1882 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:19.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.488+0000 7f0a3dffb640 1 -- 172.21.15.5:0/1209555332 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a340b0070 con 0x7f0a4013f260 2024-03-18T15:34:19.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.488+0000 7f0a3dffb640 1 -- 172.21.15.5:0/1209555332 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a340ab600 con 0x7f0a4013f260 2024-03-18T15:34:19.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.488+0000 7f0a3dffb640 1 -- 172.21.15.5:0/1209555332 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a340ab9b0 con 0x7f0a4013f260 2024-03-18T15:34:19.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.488+0000 7f0a45a12640 1 -- 172.21.15.5:0/1209555332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a4013f260 msgr2=0x7f0a4013f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:19.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.488+0000 7f0a45a12640 1 --2- 172.21.15.5:0/1209555332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a4013f260 0x7f0a4013f6a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0a340097c0 tx=0x7f0a3401c410 comp rx=0 tx=0).stop 2024-03-18T15:34:19.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.488+0000 7f0a45a12640 1 -- 172.21.15.5:0/1209555332 shutdown_connections 2024-03-18T15:34:19.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.488+0000 7f0a45a12640 1 --2- 172.21.15.5:0/1209555332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a4013f260 0x7f0a4013f6a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:19.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.488+0000 7f0a45a12640 1 -- 172.21.15.5:0/1209555332 >> 172.21.15.5:0/1209555332 conn(0x7f0a400b5670 msgr2=0x7f0a400b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:19.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.488+0000 7f0a45a12640 1 -- 172.21.15.5:0/1209555332 shutdown_connections 2024-03-18T15:34:19.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.488+0000 7f0a45a12640 1 -- 172.21.15.5:0/1209555332 wait complete. 2024-03-18T15:34:19.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.488+0000 7f0a45a12640 1 Processor -- start 2024-03-18T15:34:19.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.488+0000 7f0a45a12640 1 -- start start 2024-03-18T15:34:19.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.488+0000 7f0a45a12640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a4006d2d0 0x7f0a40071c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:19.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.488+0000 7f0a45a12640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a4013f260 0x7f0a401e3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:19.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.488+0000 7f0a45a12640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a4014dc70 0x7f0a40135930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:19.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.488+0000 7f0a45a12640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0a400b0960 con 0x7f0a4014dc70 2024-03-18T15:34:19.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.488+0000 7f0a45a12640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0a400b07e0 con 0x7f0a4006d2d0 2024-03-18T15:34:19.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.488+0000 7f0a45a12640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f0a400b5140 con 0x7f0a4013f260 2024-03-18T15:34:19.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.488+0000 7f0a3f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a4014dc70 0x7f0a40135930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:19.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.488+0000 7f0a3f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a4014dc70 0x7f0a40135930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46026/0 (socket says 172.21.15.5:46026) 2024-03-18T15:34:19.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.488+0000 7f0a3f7fe640 1 -- 172.21.15.5:0/127438399 learned_addr learned my addr 172.21.15.5:0/127438399 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:19.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.488+0000 7f0a3effd640 1 --2- 172.21.15.5:0/127438399 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a4006d2d0 0x7f0a40071c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:19.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.488+0000 7f0a3f7fe640 1 -- 172.21.15.5:0/127438399 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a4013f260 msgr2=0x7f0a401e3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:19.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.488+0000 7f0a3e7fc640 1 --2- 172.21.15.5:0/127438399 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a4013f260 0x7f0a401e3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:19.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.488+0000 7f0a3f7fe640 1 --2- 172.21.15.5:0/127438399 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a4013f260 0x7f0a401e3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:19.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.488+0000 7f0a3f7fe640 1 -- 172.21.15.5:0/127438399 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a4006d2d0 msgr2=0x7f0a40071c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:19.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.488+0000 7f0a3f7fe640 1 --2- 172.21.15.5:0/127438399 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a4006d2d0 0x7f0a40071c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:19.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.488+0000 7f0a3f7fe640 1 -- 172.21.15.5:0/127438399 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a400703d0 con 0x7f0a4014dc70 2024-03-18T15:34:19.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.488+0000 7f0a3effd640 1 --2- 172.21.15.5:0/127438399 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a4006d2d0 0x7f0a40071c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:19.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.488+0000 7f0a3f7fe640 1 --2- 172.21.15.5:0/127438399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a4014dc70 0x7f0a40135930 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0a3800c7a0 tx=0x7f0a3800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:19.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.488+0000 7f0a27fff640 1 -- 172.21.15.5:0/127438399 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a38003b80 con 0x7f0a4014dc70 2024-03-18T15:34:19.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.488+0000 7f0a27fff640 1 -- 172.21.15.5:0/127438399 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a38003d20 con 0x7f0a4014dc70 2024-03-18T15:34:19.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.488+0000 7f0a27fff640 1 -- 172.21.15.5:0/127438399 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a3800aa30 con 0x7f0a4014dc70 2024-03-18T15:34:19.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.488+0000 7f0a45a12640 1 -- 172.21.15.5:0/127438399 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a400afa50 con 0x7f0a4014dc70 2024-03-18T15:34:19.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.488+0000 7f0a45a12640 1 -- 172.21.15.5:0/127438399 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a40072190 con 0x7f0a4014dc70 2024-03-18T15:34:19.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.492+0000 7f0a45a12640 1 -- 172.21.15.5:0/127438399 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a400ae7c0 con 0x7f0a4014dc70 2024-03-18T15:34:19.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.492+0000 7f0a27fff640 1 -- 172.21.15.5:0/127438399 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f0a3800abd0 con 0x7f0a4014dc70 2024-03-18T15:34:19.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.492+0000 7f0a27fff640 1 --2- 172.21.15.5:0/127438399 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0a2c067660 0x7f0a2c069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:19.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.492+0000 7f0a3effd640 1 --2- 172.21.15.5:0/127438399 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0a2c067660 0x7f0a2c069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:19.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.492+0000 7f0a27fff640 1 -- 172.21.15.5:0/127438399 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 4857+0+0 (secure 0 0 0) 0x7f0a38085410 con 0x7f0a4014dc70 2024-03-18T15:34:19.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.492+0000 7f0a3effd640 1 --2- 172.21.15.5:0/127438399 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0a2c067660 0x7f0a2c069b20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0a300011f0 tx=0x7f0a30009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:19.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.496+0000 7f0a27fff640 1 -- 172.21.15.5:0/127438399 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f0a400ae7c0 con 0x7f0a4014dc70 2024-03-18T15:34:19.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.652+0000 7f0a45a12640 1 -- 172.21.15.5:0/127438399 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0a4013e7d0 con 0x7f0a4014dc70 2024-03-18T15:34:19.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.652+0000 7f0a27fff640 1 -- 172.21.15.5:0/127438399 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+12735 (secure 0 0 0) 0x7f0a38056470 con 0x7f0a4014dc70 2024-03-18T15:34:19.657 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:34:19.657 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"fsid":"ef54fe03-5aa7-4eb2-9b1e-66c9cc345b94","created":"2024-03-18T15:33:18.086614+0000","modified":"2024-03-18T15:34:16.692990+0000","last_up_change":"2024-03-18T15:33:29.229911+0000","last_in_change":"2024-03-18T15:33:20.323840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:28.963186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df1c754b-af59-4d16-8919-fbb6386b3c48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6804","nonce":976264754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6808","nonce":976264754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6815","nonce":976264754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6811","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6813","nonce":976264754}]},"public_addr":"172.21.15.5:6804/976264754","cluster_addr":"172.21.15.5:6808/976264754","heartbeat_back_addr":"172.21.15.5:6815/976264754","heartbeat_front_addr":"172.21.15.5:6813/976264754","state":["exists","up"]},{"osd":1,"uuid":"b9a08b5e-8af4-4144-bdba-d69ed1ac627a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6801","nonce":3901761126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6803","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6805","nonce":3901761126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6812","nonce":3901761126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6807","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6809","nonce":3901761126}]},"public_addr":"172.21.15.5:6801/3901761126","cluster_addr":"172.21.15.5:6805/3901761126","heartbeat_back_addr":"172.21.15.5:6812/3901761126","heartbeat_front_addr":"172.21.15.5:6809/3901761126","state":["exists","up"]},{"osd":2,"uuid":"c3e8bb76-e382-439d-b380-62ef92f1255a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6817","nonce":502968448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6819","nonce":502968448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6823","nonce":502968448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6821","nonce":502968448}]},"public_addr":"172.21.15.5:6817/502968448","cluster_addr":"172.21.15.5:6819/502968448","heartbeat_back_addr":"172.21.15.5:6823/502968448","heartbeat_front_addr":"172.21.15.5:6821/502968448","state":["exists","up"]},{"osd":3,"uuid":"e8c94625-f2cd-4eba-b53d-2e7356e1c02a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6825","nonce":2425932735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6827","nonce":2425932735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6831","nonce":2425932735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6829","nonce":2425932735}]},"public_addr":"172.21.15.5:6825/2425932735","cluster_addr":"172.21.15.5:6827/2425932735","heartbeat_back_addr":"172.21.15.5:6831/2425932735","heartbeat_front_addr":"172.21.15.5:6829/2425932735","state":["exists","up"]},{"osd":4,"uuid":"0099dd52-6265-4d32-82e4-3ef16e73add0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6817","nonce":976514285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6819","nonce":976514285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6823","nonce":976514285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6821","nonce":976514285}]},"public_addr":"172.21.15.80:6817/976514285","cluster_addr":"172.21.15.80:6819/976514285","heartbeat_back_addr":"172.21.15.80:6823/976514285","heartbeat_front_addr":"172.21.15.80:6821/976514285","state":["exists","up"]},{"osd":5,"uuid":"ed10c84f-8c39-41cc-9b64-7c482a94a48e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6825","nonce":627153447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6827","nonce":627153447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6831","nonce":627153447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6829","nonce":627153447}]},"public_addr":"172.21.15.80:6825/627153447","cluster_addr":"172.21.15.80:6827/627153447","heartbeat_back_addr":"172.21.15.80:6831/627153447","heartbeat_front_addr":"172.21.15.80:6829/627153447","state":["exists","up"]},{"osd":6,"uuid":"c7a3538a-7048-4fba-8781-b440f03c25db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6803","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6805","nonce":488342527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6807","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6809","nonce":488342527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6815","nonce":488342527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6811","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6813","nonce":488342527}]},"public_addr":"172.21.15.80:6805/488342527","cluster_addr":"172.21.15.80:6809/488342527","heartbeat_back_addr":"172.21.15.80:6815/488342527","heartbeat_front_addr":"172.21.15.80:6813/488342527","state":["exists","up"]},{"osd":7,"uuid":"f9418104-3a97-4110-bc3a-1f8151d0f757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6801","nonce":982831666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6804","nonce":982831666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6812","nonce":982831666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6808","nonce":982831666}]},"public_addr":"172.21.15.80:6801/982831666","cluster_addr":"172.21.15.80:6804/982831666","heartbeat_back_addr":"172.21.15.80:6812/982831666","heartbeat_front_addr":"172.21.15.80:6808/982831666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.970828+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.858116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.987556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.864603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.756293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.591502+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:34:19.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.656+0000 7f0a45a12640 1 -- 172.21.15.5:0/127438399 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0a2c067660 msgr2=0x7f0a2c069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:19.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.656+0000 7f0a45a12640 1 --2- 172.21.15.5:0/127438399 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0a2c067660 0x7f0a2c069b20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0a300011f0 tx=0x7f0a30009040 comp rx=0 tx=0).stop 2024-03-18T15:34:19.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.656+0000 7f0a45a12640 1 -- 172.21.15.5:0/127438399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a4014dc70 msgr2=0x7f0a40135930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:19.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.656+0000 7f0a45a12640 1 --2- 172.21.15.5:0/127438399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a4014dc70 0x7f0a40135930 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0a3800c7a0 tx=0x7f0a3800cc60 comp rx=0 tx=0).stop 2024-03-18T15:34:19.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.656+0000 7f0a45a12640 1 -- 172.21.15.5:0/127438399 shutdown_connections 2024-03-18T15:34:19.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.656+0000 7f0a45a12640 1 --2- 172.21.15.5:0/127438399 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0a2c067660 0x7f0a2c069b20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:19.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.656+0000 7f0a45a12640 1 --2- 172.21.15.5:0/127438399 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a4014dc70 0x7f0a40135930 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:19.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.656+0000 7f0a45a12640 1 --2- 172.21.15.5:0/127438399 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0a4013f260 0x7f0a401e3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:19.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.656+0000 7f0a45a12640 1 --2- 172.21.15.5:0/127438399 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a4006d2d0 0x7f0a40071c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:19.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.656+0000 7f0a45a12640 1 -- 172.21.15.5:0/127438399 >> 172.21.15.5:0/127438399 conn(0x7f0a400b5670 msgr2=0x7f0a4006c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:19.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.656+0000 7f0a45a12640 1 -- 172.21.15.5:0/127438399 shutdown_connections 2024-03-18T15:34:19.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.656+0000 7f0a45a12640 1 -- 172.21.15.5:0/127438399 wait complete. 2024-03-18T15:34:19.675 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T15:33:28.963186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T15:34:19.675 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T15:34:19.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.796+0000 7f3fe779c640 1 Processor -- start 2024-03-18T15:34:19.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.796+0000 7f3fe779c640 1 -- start start 2024-03-18T15:34:19.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.796+0000 7f3fe779c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3fe00af790 0x7f3fe0072b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:19.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.796+0000 7f3fe779c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3fe006d070 0x7f3fe0147780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:19.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.796+0000 7f3fe779c640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3fe00b26a0 0x7f3fe00b2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:19.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.796+0000 7f3fe779c640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f3fe00b3070 con 0x7f3fe006d070 2024-03-18T15:34:19.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.796+0000 7f3fe779c640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f3fe00afb70 con 0x7f3fe00af790 2024-03-18T15:34:19.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.796+0000 7f3fe779c640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f3fe00b4a20 con 0x7f3fe00b26a0 2024-03-18T15:34:19.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.796+0000 7f3fe4d37640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3fe006d070 0x7f3fe0147780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:19.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.796+0000 7f3fe4d37640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3fe006d070 0x7f3fe0147780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46028/0 (socket says 172.21.15.5:46028) 2024-03-18T15:34:19.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.796+0000 7f3fe4d37640 1 -- 172.21.15.5:0/1781656288 learned_addr learned my addr 172.21.15.5:0/1781656288 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:19.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.800+0000 7f3fe5d39640 1 --2- 172.21.15.5:0/1781656288 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3fe00b26a0 0x7f3fe00b2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:19.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.800+0000 7f3fe4d37640 1 -- 172.21.15.5:0/1781656288 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3fe00b26a0 msgr2=0x7f3fe00b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:19.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.800+0000 7f3fe5538640 1 --2- 172.21.15.5:0/1781656288 >> v2:172.21.15.80:3300/0 conn(0x7f3fe00af790 0x7f3fe0072b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:19.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.800+0000 7f3fe4d37640 1 --2- 172.21.15.5:0/1781656288 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3fe00b26a0 0x7f3fe00b2b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:19.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.800+0000 7f3fe4d37640 1 -- 172.21.15.5:0/1781656288 >> v2:172.21.15.80:3300/0 conn(0x7f3fe00af790 msgr2=0x7f3fe0072b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:19.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.800+0000 7f3fe4d37640 1 --2- 172.21.15.5:0/1781656288 >> v2:172.21.15.80:3300/0 conn(0x7f3fe00af790 0x7f3fe0072b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:19.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.800+0000 7f3fe4d37640 1 -- 172.21.15.5:0/1781656288 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fe00b4d00 con 0x7f3fe006d070 2024-03-18T15:34:19.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.800+0000 7f3fe4d37640 1 --2- 172.21.15.5:0/1781656288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe006d070 0x7f3fe0147780 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3fe006dd00 tx=0x7f3fd001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2671d057da83aa86 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:19.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.800+0000 7f3fd7fff640 1 -- 172.21.15.5:0/1781656288 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd00b2070 con 0x7f3fe006d070 2024-03-18T15:34:19.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.800+0000 7f3fd7fff640 1 -- 172.21.15.5:0/1781656288 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fd00a9070 con 0x7f3fe006d070 2024-03-18T15:34:19.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.800+0000 7f3fd7fff640 1 -- 172.21.15.5:0/1781656288 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd00ad880 con 0x7f3fe006d070 2024-03-18T15:34:19.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.800+0000 7f3fe779c640 1 -- 172.21.15.5:0/1781656288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe006d070 msgr2=0x7f3fe0147780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:19.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.800+0000 7f3fe779c640 1 --2- 172.21.15.5:0/1781656288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe006d070 0x7f3fe0147780 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3fe006dd00 tx=0x7f3fd001c490 comp rx=0 tx=0).stop 2024-03-18T15:34:19.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.800+0000 7f3fe779c640 1 -- 172.21.15.5:0/1781656288 shutdown_connections 2024-03-18T15:34:19.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.800+0000 7f3fe779c640 1 --2- 172.21.15.5:0/1781656288 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3fe00b26a0 0x7f3fe00b2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:19.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.800+0000 7f3fe779c640 1 --2- 172.21.15.5:0/1781656288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe006d070 0x7f3fe0147780 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:19.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.800+0000 7f3fe779c640 1 --2- 172.21.15.5:0/1781656288 >> v2:172.21.15.80:3300/0 conn(0x7f3fe00af790 0x7f3fe0072b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:19.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.800+0000 7f3fe779c640 1 -- 172.21.15.5:0/1781656288 >> 172.21.15.5:0/1781656288 conn(0x7f3fe00b5670 msgr2=0x7f3fe00b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:19.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.800+0000 7f3fe779c640 1 -- 172.21.15.5:0/1781656288 shutdown_connections 2024-03-18T15:34:19.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.800+0000 7f3fe779c640 1 -- 172.21.15.5:0/1781656288 wait complete. 2024-03-18T15:34:19.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.804+0000 7f3fe779c640 1 Processor -- start 2024-03-18T15:34:19.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.804+0000 7f3fe779c640 1 -- start start 2024-03-18T15:34:19.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.804+0000 7f3fe779c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe006d070 0x7f3fe013ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:19.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.804+0000 7f3fe779c640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3fe00af790 0x7f3fe0070ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:19.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.804+0000 7f3fe779c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3fe00b26a0 0x7f3fe0071780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:19.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.804+0000 7f3fe779c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe0144b60 con 0x7f3fe006d070 2024-03-18T15:34:19.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.804+0000 7f3fe779c640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe01449e0 con 0x7f3fe00b26a0 2024-03-18T15:34:19.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.804+0000 7f3fe779c640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f3fe0144ce0 con 0x7f3fe00af790 2024-03-18T15:34:19.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.804+0000 7f3fe5538640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe006d070 0x7f3fe013ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:19.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.804+0000 7f3fe5538640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe006d070 0x7f3fe013ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46030/0 (socket says 172.21.15.5:46030) 2024-03-18T15:34:19.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.804+0000 7f3fe5538640 1 -- 172.21.15.5:0/3773172379 learned_addr learned my addr 172.21.15.5:0/3773172379 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:19.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.804+0000 7f3fe4d37640 1 --2- 172.21.15.5:0/3773172379 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3fe00af790 0x7f3fe0070ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:19.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.804+0000 7f3fe5538640 1 -- 172.21.15.5:0/3773172379 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3fe00af790 msgr2=0x7f3fe0070ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:19.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.804+0000 7f3fe5538640 1 --2- 172.21.15.5:0/3773172379 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3fe00af790 0x7f3fe0070ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:19.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.804+0000 7f3fe5538640 1 -- 172.21.15.5:0/3773172379 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3fe00b26a0 msgr2=0x7f3fe0071780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:19.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.804+0000 7f3fe5538640 1 --2- 172.21.15.5:0/3773172379 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3fe00b26a0 0x7f3fe0071780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:19.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.804+0000 7f3fe5538640 1 -- 172.21.15.5:0/3773172379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fe0071000 con 0x7f3fe006d070 2024-03-18T15:34:19.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.804+0000 7f3fe5538640 1 --2- 172.21.15.5:0/3773172379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe006d070 0x7f3fe013ae20 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc00c7f0 tx=0x7f3fdc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:19.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.804+0000 7f3fd5ffb640 1 -- 172.21.15.5:0/3773172379 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc015070 con 0x7f3fe006d070 2024-03-18T15:34:19.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.804+0000 7f3fd5ffb640 1 -- 172.21.15.5:0/3773172379 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fdc004220 con 0x7f3fe006d070 2024-03-18T15:34:19.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.804+0000 7f3fe779c640 1 -- 172.21.15.5:0/3773172379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fe00721b0 con 0x7f3fe006d070 2024-03-18T15:34:19.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.804+0000 7f3fd5ffb640 1 -- 172.21.15.5:0/3773172379 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc003620 con 0x7f3fe006d070 2024-03-18T15:34:19.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.804+0000 7f3fe779c640 1 -- 172.21.15.5:0/3773172379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fe00712d0 con 0x7f3fe006d070 2024-03-18T15:34:19.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.804+0000 7f3fe779c640 1 -- 172.21.15.5:0/3773172379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fe004f550 con 0x7f3fe006d070 2024-03-18T15:34:19.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.804+0000 7f3fd5ffb640 1 -- 172.21.15.5:0/3773172379 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f3fdc0039d0 con 0x7f3fe006d070 2024-03-18T15:34:19.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.804+0000 7f3fd5ffb640 1 --2- 172.21.15.5:0/3773172379 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3fc4067660 0x7f3fc4069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:19.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.804+0000 7f3fe4d37640 1 --2- 172.21.15.5:0/3773172379 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3fc4067660 0x7f3fc4069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:19.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.804+0000 7f3fd5ffb640 1 -- 172.21.15.5:0/3773172379 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 4857+0+0 (secure 0 0 0) 0x7f3fdc0855a0 con 0x7f3fe006d070 2024-03-18T15:34:19.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.808+0000 7f3fe4d37640 1 --2- 172.21.15.5:0/3773172379 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3fc4067660 0x7f3fc4069b20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3fd00af000 tx=0x7f3fd0004740 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:19.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.808+0000 7f3fd5ffb640 1 -- 172.21.15.5:0/3773172379 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f3fdc056560 con 0x7f3fe006d070 2024-03-18T15:34:19.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.972+0000 7f3fe779c640 1 -- 172.21.15.5:0/3773172379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3fe00ae840 con 0x7f3fe006d070 2024-03-18T15:34:19.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.976+0000 7f3fd5ffb640 1 -- 172.21.15.5:0/3773172379 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) ==== 93+0+10 (secure 0 0 0) 0x7f3fdc055e60 con 0x7f3fe006d070 2024-03-18T15:34:19.978 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-03-18T15:34:19.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.976+0000 7f3fe779c640 1 -- 172.21.15.5:0/3773172379 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3fc4067660 msgr2=0x7f3fc4069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:19.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.976+0000 7f3fe779c640 1 --2- 172.21.15.5:0/3773172379 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3fc4067660 0x7f3fc4069b20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3fd00af000 tx=0x7f3fd0004740 comp rx=0 tx=0).stop 2024-03-18T15:34:19.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.976+0000 7f3fe779c640 1 -- 172.21.15.5:0/3773172379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe006d070 msgr2=0x7f3fe013ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:19.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.976+0000 7f3fe779c640 1 --2- 172.21.15.5:0/3773172379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe006d070 0x7f3fe013ae20 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc00c7f0 tx=0x7f3fdc00ccb0 comp rx=0 tx=0).stop 2024-03-18T15:34:19.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.976+0000 7f3fe779c640 1 -- 172.21.15.5:0/3773172379 shutdown_connections 2024-03-18T15:34:19.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.976+0000 7f3fe779c640 1 --2- 172.21.15.5:0/3773172379 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3fc4067660 0x7f3fc4069b20 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:19.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.976+0000 7f3fe779c640 1 --2- 172.21.15.5:0/3773172379 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3fe00b26a0 0x7f3fe0071780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:19.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.976+0000 7f3fe779c640 1 --2- 172.21.15.5:0/3773172379 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3fe00af790 0x7f3fe0070ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:19.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.976+0000 7f3fe779c640 1 --2- 172.21.15.5:0/3773172379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fe006d070 0x7f3fe013ae20 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:19.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.976+0000 7f3fe779c640 1 -- 172.21.15.5:0/3773172379 >> 172.21.15.5:0/3773172379 conn(0x7f3fe00b5670 msgr2=0x7f3fe00aed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:19.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.976+0000 7f3fe779c640 1 -- 172.21.15.5:0/3773172379 shutdown_connections 2024-03-18T15:34:19.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:19.976+0000 7f3fe779c640 1 -- 172.21.15.5:0/3773172379 wait complete. 2024-03-18T15:34:19.996 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-18T15:34:20.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.112+0000 7f04d9eeb640 1 Processor -- start 2024-03-18T15:34:20.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.112+0000 7f04d9eeb640 1 -- start start 2024-03-18T15:34:20.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.116+0000 7f04d9eeb640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f04d40b9d50 0x7f04d40b44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:20.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.116+0000 7f04d9eeb640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f04d414a940 0x7f04d41412f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:20.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.116+0000 7f04d9eeb640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f04d40b4dd0 0x7f04d4141830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:20.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.116+0000 7f04d9eeb640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f04d40b4a00 con 0x7f04d40b9d50 2024-03-18T15:34:20.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.116+0000 7f04d9eeb640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f04d40ba130 con 0x7f04d40b4dd0 2024-03-18T15:34:20.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.116+0000 7f04d9eeb640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f04d40b53b0 con 0x7f04d414a940 2024-03-18T15:34:20.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.116+0000 7f04d37fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f04d40b9d50 0x7f04d40b44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:20.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.116+0000 7f04d37fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f04d40b9d50 0x7f04d40b44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46042/0 (socket says 172.21.15.5:46042) 2024-03-18T15:34:20.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.116+0000 7f04d37fe640 1 -- 172.21.15.5:0/4187798718 learned_addr learned my addr 172.21.15.5:0/4187798718 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:20.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.116+0000 7f04d2ffd640 1 --2- 172.21.15.5:0/4187798718 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f04d414a940 0x7f04d41412f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:20.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.116+0000 7f04d3fff640 1 --2- 172.21.15.5:0/4187798718 >> v2:172.21.15.80:3300/0 conn(0x7f04d40b4dd0 0x7f04d4141830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:20.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.116+0000 7f04d37fe640 1 -- 172.21.15.5:0/4187798718 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f04d414a940 msgr2=0x7f04d41412f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:20.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.116+0000 7f04d37fe640 1 --2- 172.21.15.5:0/4187798718 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f04d414a940 0x7f04d41412f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:20.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.116+0000 7f04d37fe640 1 -- 172.21.15.5:0/4187798718 >> v2:172.21.15.80:3300/0 conn(0x7f04d40b4dd0 msgr2=0x7f04d4141830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:20.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.116+0000 7f04d37fe640 1 --2- 172.21.15.5:0/4187798718 >> v2:172.21.15.80:3300/0 conn(0x7f04d40b4dd0 0x7f04d4141830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:20.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.116+0000 7f04d37fe640 1 -- 172.21.15.5:0/4187798718 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04d40b5690 con 0x7f04d40b9d50 2024-03-18T15:34:20.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.116+0000 7f04d37fe640 1 --2- 172.21.15.5:0/4187798718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04d40b9d50 0x7f04d40b44c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f04d40b3d20 tx=0x7f04c401c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a5a49bd71d5f805 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:20.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.116+0000 7f04d27fc640 1 -- 172.21.15.5:0/4187798718 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c40a5830 con 0x7f04d40b9d50 2024-03-18T15:34:20.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.116+0000 7f04d27fc640 1 -- 172.21.15.5:0/4187798718 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04c40a59d0 con 0x7f04d40b9d50 2024-03-18T15:34:20.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.116+0000 7f04d27fc640 1 -- 172.21.15.5:0/4187798718 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c40a5da0 con 0x7f04d40b9d50 2024-03-18T15:34:20.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.120+0000 7f04d9eeb640 1 -- 172.21.15.5:0/4187798718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04d40b9d50 msgr2=0x7f04d40b44c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:20.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.120+0000 7f04d9eeb640 1 --2- 172.21.15.5:0/4187798718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04d40b9d50 0x7f04d40b44c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f04d40b3d20 tx=0x7f04c401c990 comp rx=0 tx=0).stop 2024-03-18T15:34:20.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.120+0000 7f04d9eeb640 1 -- 172.21.15.5:0/4187798718 shutdown_connections 2024-03-18T15:34:20.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.120+0000 7f04d9eeb640 1 --2- 172.21.15.5:0/4187798718 >> v2:172.21.15.80:3300/0 conn(0x7f04d40b4dd0 0x7f04d4141830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:20.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.120+0000 7f04d9eeb640 1 --2- 172.21.15.5:0/4187798718 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f04d414a940 0x7f04d41412f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:20.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.120+0000 7f04d9eeb640 1 --2- 172.21.15.5:0/4187798718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04d40b9d50 0x7f04d40b44c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:20.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.120+0000 7f04d9eeb640 1 -- 172.21.15.5:0/4187798718 >> 172.21.15.5:0/4187798718 conn(0x7f04d4067190 msgr2=0x7f04d40675a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:20.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.120+0000 7f04d9eeb640 1 -- 172.21.15.5:0/4187798718 shutdown_connections 2024-03-18T15:34:20.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.120+0000 7f04d9eeb640 1 -- 172.21.15.5:0/4187798718 wait complete. 2024-03-18T15:34:20.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.120+0000 7f04d9eeb640 1 Processor -- start 2024-03-18T15:34:20.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.120+0000 7f04d9eeb640 1 -- start start 2024-03-18T15:34:20.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.120+0000 7f04d9eeb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04d40b4dd0 0x7f04d41427d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:20.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.120+0000 7f04d9eeb640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04d40b9d50 0x7f04d41e0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:20.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.120+0000 7f04d9eeb640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f04d414a940 0x7f04d41e0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:20.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.120+0000 7f04d9eeb640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f04d4147f50 con 0x7f04d40b4dd0 2024-03-18T15:34:20.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.120+0000 7f04d9eeb640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f04d4147dd0 con 0x7f04d40b9d50 2024-03-18T15:34:20.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.120+0000 7f04d9eeb640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f04d41480d0 con 0x7f04d414a940 2024-03-18T15:34:20.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.120+0000 7f04d37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04d40b4dd0 0x7f04d41427d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:20.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.120+0000 7f04d37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04d40b4dd0 0x7f04d41427d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46046/0 (socket says 172.21.15.5:46046) 2024-03-18T15:34:20.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.120+0000 7f04d37fe640 1 -- 172.21.15.5:0/1197210369 learned_addr learned my addr 172.21.15.5:0/1197210369 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:20.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.120+0000 7f04d3fff640 1 --2- 172.21.15.5:0/1197210369 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f04d414a940 0x7f04d41e0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:20.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.120+0000 7f04d2ffd640 1 --2- 172.21.15.5:0/1197210369 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04d40b9d50 0x7f04d41e0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:20.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.120+0000 7f04d37fe640 1 -- 172.21.15.5:0/1197210369 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f04d414a940 msgr2=0x7f04d41e0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:20.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.120+0000 7f04d37fe640 1 --2- 172.21.15.5:0/1197210369 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f04d414a940 0x7f04d41e0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:20.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.120+0000 7f04d37fe640 1 -- 172.21.15.5:0/1197210369 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04d40b9d50 msgr2=0x7f04d41e0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:20.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.120+0000 7f04d37fe640 1 --2- 172.21.15.5:0/1197210369 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04d40b9d50 0x7f04d41e0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:20.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.120+0000 7f04d37fe640 1 -- 172.21.15.5:0/1197210369 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04d4074950 con 0x7f04d40b4dd0 2024-03-18T15:34:20.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.120+0000 7f04d2ffd640 1 --2- 172.21.15.5:0/1197210369 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04d40b9d50 0x7f04d41e0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:20.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.120+0000 7f04d37fe640 1 --2- 172.21.15.5:0/1197210369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04d40b4dd0 0x7f04d41427d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f04c401d230 tx=0x7f04d41483c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:20.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.120+0000 7f04bbfff640 1 -- 172.21.15.5:0/1197210369 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c40bc030 con 0x7f04d40b4dd0 2024-03-18T15:34:20.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.120+0000 7f04bbfff640 1 -- 172.21.15.5:0/1197210369 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04c4009250 con 0x7f04d40b4dd0 2024-03-18T15:34:20.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.120+0000 7f04d9eeb640 1 -- 172.21.15.5:0/1197210369 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04d40b1120 con 0x7f04d40b4dd0 2024-03-18T15:34:20.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.120+0000 7f04bbfff640 1 -- 172.21.15.5:0/1197210369 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c40b88c0 con 0x7f04d40b4dd0 2024-03-18T15:34:20.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.120+0000 7f04d9eeb640 1 -- 172.21.15.5:0/1197210369 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04d4143fb0 con 0x7f04d40b4dd0 2024-03-18T15:34:20.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.120+0000 7f04d9eeb640 1 -- 172.21.15.5:0/1197210369 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04d404f550 con 0x7f04d40b4dd0 2024-03-18T15:34:20.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.124+0000 7f04bbfff640 1 -- 172.21.15.5:0/1197210369 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f04c40bf070 con 0x7f04d40b4dd0 2024-03-18T15:34:20.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.124+0000 7f04bbfff640 1 --2- 172.21.15.5:0/1197210369 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f04c0067660 0x7f04c0069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:20.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.124+0000 7f04bbfff640 1 -- 172.21.15.5:0/1197210369 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 4857+0+0 (secure 0 0 0) 0x7f04c4120000 con 0x7f04d40b4dd0 2024-03-18T15:34:20.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.124+0000 7f04d2ffd640 1 --2- 172.21.15.5:0/1197210369 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f04c0067660 0x7f04c0069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:20.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.124+0000 7f04d2ffd640 1 --2- 172.21.15.5:0/1197210369 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f04c0067660 0x7f04c0069b20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f04d4067670 tx=0x7f04c8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:20.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.128+0000 7f04bbfff640 1 -- 172.21.15.5:0/1197210369 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f04c40c7020 con 0x7f04d40b4dd0 2024-03-18T15:34:20.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.288+0000 7f04d9eeb640 1 -- 172.21.15.5:0/1197210369 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f04d40b1390 con 0x7f04d40b4dd0 2024-03-18T15:34:20.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.292+0000 7f04bbfff640 1 -- 172.21.15.5:0/1197210369 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v26) ==== 84+0+65 (secure 0 0 0) 0x7f04c40165e0 con 0x7f04d40b4dd0 2024-03-18T15:34:20.293 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:34:20.293 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2024-03-18T15:34:20.294 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-03-18T15:34:20.294 INFO:teuthology.orchestra.run.smithi005.stdout: "poolnum": 1, 2024-03-18T15:34:20.294 INFO:teuthology.orchestra.run.smithi005.stdout: "poolname": ".mgr" 2024-03-18T15:34:20.294 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-03-18T15:34:20.294 INFO:teuthology.orchestra.run.smithi005.stdout:] 2024-03-18T15:34:20.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.292+0000 7f04d9eeb640 1 -- 172.21.15.5:0/1197210369 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f04c0067660 msgr2=0x7f04c0069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:20.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.292+0000 7f04d9eeb640 1 --2- 172.21.15.5:0/1197210369 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f04c0067660 0x7f04c0069b20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f04d4067670 tx=0x7f04c8008040 comp rx=0 tx=0).stop 2024-03-18T15:34:20.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.292+0000 7f04d9eeb640 1 -- 172.21.15.5:0/1197210369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04d40b4dd0 msgr2=0x7f04d41427d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:20.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.292+0000 7f04d9eeb640 1 --2- 172.21.15.5:0/1197210369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04d40b4dd0 0x7f04d41427d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f04c401d230 tx=0x7f04d41483c0 comp rx=0 tx=0).stop 2024-03-18T15:34:20.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.292+0000 7f04d9eeb640 1 -- 172.21.15.5:0/1197210369 shutdown_connections 2024-03-18T15:34:20.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.292+0000 7f04d9eeb640 1 --2- 172.21.15.5:0/1197210369 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f04c0067660 0x7f04c0069b20 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:20.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.292+0000 7f04d9eeb640 1 --2- 172.21.15.5:0/1197210369 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f04d414a940 0x7f04d41e0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:20.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.292+0000 7f04d9eeb640 1 --2- 172.21.15.5:0/1197210369 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04d40b9d50 0x7f04d41e0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:20.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.292+0000 7f04d9eeb640 1 --2- 172.21.15.5:0/1197210369 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04d40b4dd0 0x7f04d41427d0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:20.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.292+0000 7f04d9eeb640 1 -- 172.21.15.5:0/1197210369 >> 172.21.15.5:0/1197210369 conn(0x7f04d4067190 msgr2=0x7f04d40b75f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:20.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.292+0000 7f04d9eeb640 1 -- 172.21.15.5:0/1197210369 shutdown_connections 2024-03-18T15:34:20.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.292+0000 7f04d9eeb640 1 -- 172.21.15.5:0/1197210369 wait complete. 2024-03-18T15:34:20.312 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-18T15:34:20.312 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-18T15:34:20.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.432+0000 7f293822c640 1 Processor -- start 2024-03-18T15:34:20.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.432+0000 7f293822c640 1 -- start start 2024-03-18T15:34:20.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.436+0000 7f293822c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f29301471c0 0x7f29300b0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:20.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.436+0000 7f293822c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f29300b2410 0x7f29301479e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:20.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.436+0000 7f293822c640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29301475a0 con 0x7f29300b08b0 2024-03-18T15:34:20.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.436+0000 7f293822c640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f29300b17f0 con 0x7f29301471c0 2024-03-18T15:34:20.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.436+0000 7f293822c640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f29300b2c40 con 0x7f29300b2410 2024-03-18T15:34:20.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.436+0000 7f2935fc8640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f29301471c0 0x7f29300b0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:20.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.436+0000 7f2935fc8640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f29301471c0 0x7f29300b0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46054/0 (socket says 172.21.15.5:46054) 2024-03-18T15:34:20.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.436+0000 7f2935fc8640 1 -- 172.21.15.5:0/943214982 learned_addr learned my addr 172.21.15.5:0/943214982 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:20.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.436+0000 7f29357c7640 1 --2- 172.21.15.5:0/943214982 >> v2:172.21.15.80:3300/0 conn(0x7f29300b2410 0x7f29301479e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:20.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.436+0000 7f2935fc8640 1 -- 172.21.15.5:0/943214982 >> v2:172.21.15.80:3300/0 conn(0x7f29300b2410 msgr2=0x7f29301479e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:20.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.436+0000 7f2935fc8640 1 --2- 172.21.15.5:0/943214982 >> v2:172.21.15.80:3300/0 conn(0x7f29300b2410 0x7f29301479e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:20.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.436+0000 7f2935fc8640 1 -- 172.21.15.5:0/943214982 >> v1:172.21.15.80:6789/0 conn(0x7f29300b08b0 legacy=0x7f29300b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:20.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.436+0000 7f2935fc8640 1 -- 172.21.15.5:0/943214982 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29300b2f20 con 0x7f29301471c0 2024-03-18T15:34:20.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.436+0000 7f2935fc8640 1 --2- 172.21.15.5:0/943214982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29301471c0 0x7f29300b0070 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f293006db80 tx=0x7f292c01e840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=454bf583e6989030 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:20.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.436+0000 7f2934fc6640 1 -- 172.21.15.5:0/943214982 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292c0ad070 con 0x7f29301471c0 2024-03-18T15:34:20.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.436+0000 7f2934fc6640 1 -- 172.21.15.5:0/943214982 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f292c004230 con 0x7f29301471c0 2024-03-18T15:34:20.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.436+0000 7f2934fc6640 1 -- 172.21.15.5:0/943214982 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292c0a7690 con 0x7f29301471c0 2024-03-18T15:34:20.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.436+0000 7f293822c640 1 -- 172.21.15.5:0/943214982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29301471c0 msgr2=0x7f29300b0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:20.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.436+0000 7f293822c640 1 --2- 172.21.15.5:0/943214982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29301471c0 0x7f29300b0070 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f293006db80 tx=0x7f292c01e840 comp rx=0 tx=0).stop 2024-03-18T15:34:20.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.436+0000 7f293822c640 1 -- 172.21.15.5:0/943214982 shutdown_connections 2024-03-18T15:34:20.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.436+0000 7f293822c640 1 --2- 172.21.15.5:0/943214982 >> v2:172.21.15.80:3300/0 conn(0x7f29300b2410 0x7f29301479e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:20.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.436+0000 7f293822c640 1 --2- 172.21.15.5:0/943214982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29301471c0 0x7f29300b0070 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:20.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.436+0000 7f293822c640 1 -- 172.21.15.5:0/943214982 >> 172.21.15.5:0/943214982 conn(0x7f29300b5670 msgr2=0x7f29300b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:20.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.436+0000 7f293822c640 1 -- 172.21.15.5:0/943214982 shutdown_connections 2024-03-18T15:34:20.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.436+0000 7f293822c640 1 -- 172.21.15.5:0/943214982 wait complete. 2024-03-18T15:34:20.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.440+0000 7f293822c640 1 Processor -- start 2024-03-18T15:34:20.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.440+0000 7f293822c640 1 -- start start 2024-03-18T15:34:20.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.440+0000 7f293822c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29300b08b0 0x7f2930146410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:20.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.440+0000 7f293822c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29300b2410 0x7f29300ae370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:20.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.440+0000 7f293822c640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f29301471c0 0x7f29300ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:20.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.440+0000 7f293822c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f29301502a0 con 0x7f29300b2410 2024-03-18T15:34:20.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.440+0000 7f293822c640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f29300b28c0 con 0x7f29300b08b0 2024-03-18T15:34:20.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.440+0000 7f293822c640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f2930150420 con 0x7f29301471c0 2024-03-18T15:34:20.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.440+0000 7f29357c7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29300b2410 0x7f29300ae370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:20.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.440+0000 7f29357c7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29300b2410 0x7f29300ae370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46066/0 (socket says 172.21.15.5:46066) 2024-03-18T15:34:20.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.440+0000 7f29357c7640 1 -- 172.21.15.5:0/1278323703 learned_addr learned my addr 172.21.15.5:0/1278323703 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:20.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.440+0000 7f29367c9640 1 --2- 172.21.15.5:0/1278323703 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f29301471c0 0x7f29300ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:20.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.440+0000 7f29357c7640 1 -- 172.21.15.5:0/1278323703 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f29301471c0 msgr2=0x7f29300ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:20.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.440+0000 7f2935fc8640 1 --2- 172.21.15.5:0/1278323703 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29300b08b0 0x7f2930146410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:20.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.440+0000 7f29357c7640 1 --2- 172.21.15.5:0/1278323703 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f29301471c0 0x7f29300ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:20.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.440+0000 7f29357c7640 1 -- 172.21.15.5:0/1278323703 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29300b08b0 msgr2=0x7f2930146410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:20.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.440+0000 7f29357c7640 1 --2- 172.21.15.5:0/1278323703 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29300b08b0 0x7f2930146410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:20.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.440+0000 7f29357c7640 1 -- 172.21.15.5:0/1278323703 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29301e3470 con 0x7f29300b2410 2024-03-18T15:34:20.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.440+0000 7f29357c7640 1 --2- 172.21.15.5:0/1278323703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29300b2410 0x7f29300ae370 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f292000c7f0 tx=0x7f292000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:20.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.440+0000 7f29267fc640 1 -- 172.21.15.5:0/1278323703 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2920015070 con 0x7f29300b2410 2024-03-18T15:34:20.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.440+0000 7f29267fc640 1 -- 172.21.15.5:0/1278323703 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2920004220 con 0x7f29300b2410 2024-03-18T15:34:20.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.440+0000 7f293822c640 1 -- 172.21.15.5:0/1278323703 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29301e3750 con 0x7f29300b2410 2024-03-18T15:34:20.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.440+0000 7f29267fc640 1 -- 172.21.15.5:0/1278323703 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2920003620 con 0x7f29300b2410 2024-03-18T15:34:20.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.440+0000 7f293822c640 1 -- 172.21.15.5:0/1278323703 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f293013d070 con 0x7f29300b2410 2024-03-18T15:34:20.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.440+0000 7f293822c640 1 -- 172.21.15.5:0/1278323703 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f293004f5c0 con 0x7f29300b2410 2024-03-18T15:34:20.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.440+0000 7f29267fc640 1 -- 172.21.15.5:0/1278323703 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f29200039d0 con 0x7f29300b2410 2024-03-18T15:34:20.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.444+0000 7f29267fc640 1 --2- 172.21.15.5:0/1278323703 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f2914067660 0x7f2914069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:20.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.444+0000 7f29267fc640 1 -- 172.21.15.5:0/1278323703 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 4857+0+0 (secure 0 0 0) 0x7f2920085550 con 0x7f29300b2410 2024-03-18T15:34:20.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.444+0000 7f2935fc8640 1 --2- 172.21.15.5:0/1278323703 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f2914067660 0x7f2914069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:20.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.444+0000 7f2935fc8640 1 --2- 172.21.15.5:0/1278323703 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f2914067660 0x7f2914069b20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f292c0b1000 tx=0x7f292c0a9070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:20.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.448+0000 7f29267fc640 1 -- 172.21.15.5:0/1278323703 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f2920056510 con 0x7f29300b2410 2024-03-18T15:34:20.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.608+0000 7f293822c640 1 -- 172.21.15.5:0/1278323703 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f29300b7390 con 0x7f29300b2410 2024-03-18T15:34:20.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.720+0000 7f29267fc640 1 -- 172.21.15.5:0/1278323703 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v27) ==== 138+0+0 (secure 0 0 0) 0x7f2920055e10 con 0x7f29300b2410 2024-03-18T15:34:20.723 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' created 2024-03-18T15:34:20.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.724+0000 7f293822c640 1 -- 172.21.15.5:0/1278323703 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f2914067660 msgr2=0x7f2914069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:20.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.724+0000 7f293822c640 1 --2- 172.21.15.5:0/1278323703 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f2914067660 0x7f2914069b20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f292c0b1000 tx=0x7f292c0a9070 comp rx=0 tx=0).stop 2024-03-18T15:34:20.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.724+0000 7f293822c640 1 -- 172.21.15.5:0/1278323703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29300b2410 msgr2=0x7f29300ae370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:20.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.724+0000 7f293822c640 1 --2- 172.21.15.5:0/1278323703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29300b2410 0x7f29300ae370 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f292000c7f0 tx=0x7f292000ccb0 comp rx=0 tx=0).stop 2024-03-18T15:34:20.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.724+0000 7f293822c640 1 -- 172.21.15.5:0/1278323703 shutdown_connections 2024-03-18T15:34:20.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.724+0000 7f293822c640 1 --2- 172.21.15.5:0/1278323703 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f2914067660 0x7f2914069b20 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:20.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.724+0000 7f293822c640 1 --2- 172.21.15.5:0/1278323703 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f29301471c0 0x7f29300ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:20.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.724+0000 7f293822c640 1 --2- 172.21.15.5:0/1278323703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f29300b2410 0x7f29300ae370 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:20.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.724+0000 7f293822c640 1 --2- 172.21.15.5:0/1278323703 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29300b08b0 0x7f2930146410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:20.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.724+0000 7f293822c640 1 -- 172.21.15.5:0/1278323703 >> 172.21.15.5:0/1278323703 conn(0x7f29300b5670 msgr2=0x7f293013bba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:20.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.724+0000 7f293822c640 1 -- 172.21.15.5:0/1278323703 shutdown_connections 2024-03-18T15:34:20.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.724+0000 7f293822c640 1 -- 172.21.15.5:0/1278323703 wait complete. 2024-03-18T15:34:20.752 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-18T15:34:20.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.888+0000 7f10882cc640 1 Processor -- start 2024-03-18T15:34:20.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.888+0000 7f10882cc640 1 -- start start 2024-03-18T15:34:20.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.888+0000 7f10882cc640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f108006d2d0 0x7f10800b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:20.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.888+0000 7f10882cc640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f108013f260 0x7f108013f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:20.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.888+0000 7f10882cc640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f10801453a0 0x7f1080145810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:20.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.888+0000 7f10882cc640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f10800728c0 con 0x7f10801453a0 2024-03-18T15:34:20.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.888+0000 7f10882cc640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f10800b0ea0 con 0x7f108013f260 2024-03-18T15:34:20.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.888+0000 7f10882cc640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f108006d6b0 con 0x7f108006d2d0 2024-03-18T15:34:20.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.888+0000 7f1086869640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f10801453a0 0x7f1080145810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:20.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.888+0000 7f1086869640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f10801453a0 0x7f1080145810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44950/0 (socket says 172.21.15.5:44950) 2024-03-18T15:34:20.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.888+0000 7f1086869640 1 -- 172.21.15.5:0/580091926 learned_addr learned my addr 172.21.15.5:0/580091926 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:20.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.888+0000 7f1085867640 1 --2- 172.21.15.5:0/580091926 >> v2:172.21.15.80:3300/0 conn(0x7f108013f260 0x7f108013f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:20.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.888+0000 7f1086068640 1 --2- 172.21.15.5:0/580091926 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f108006d2d0 0x7f10800b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:20.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.888+0000 7f1086869640 1 -- 172.21.15.5:0/580091926 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f108006d2d0 msgr2=0x7f10800b23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:20.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.888+0000 7f1086869640 1 --2- 172.21.15.5:0/580091926 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f108006d2d0 0x7f10800b23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:20.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.888+0000 7f1086869640 1 -- 172.21.15.5:0/580091926 >> v2:172.21.15.80:3300/0 conn(0x7f108013f260 msgr2=0x7f108013f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:20.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.888+0000 7f1086869640 1 --2- 172.21.15.5:0/580091926 >> v2:172.21.15.80:3300/0 conn(0x7f108013f260 0x7f108013f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:20.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.888+0000 7f1086869640 1 -- 172.21.15.5:0/580091926 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10800b0bc0 con 0x7f10801453a0 2024-03-18T15:34:20.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.888+0000 7f1085867640 1 --2- 172.21.15.5:0/580091926 >> v2:172.21.15.80:3300/0 conn(0x7f108013f260 0x7f108013f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:20.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.892+0000 7f1086869640 1 --2- 172.21.15.5:0/580091926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10801453a0 0x7f1080145810 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f10800b9ab0 tx=0x7f107801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41cf2b2a88fa83e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:20.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.892+0000 7f1085066640 1 -- 172.21.15.5:0/580091926 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10780b2070 con 0x7f10801453a0 2024-03-18T15:34:20.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.892+0000 7f1085066640 1 -- 172.21.15.5:0/580091926 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10780a9070 con 0x7f10801453a0 2024-03-18T15:34:20.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.892+0000 7f1085066640 1 -- 172.21.15.5:0/580091926 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10780ad880 con 0x7f10801453a0 2024-03-18T15:34:20.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.892+0000 7f10882cc640 1 -- 172.21.15.5:0/580091926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10801453a0 msgr2=0x7f1080145810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:20.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.892+0000 7f10882cc640 1 --2- 172.21.15.5:0/580091926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10801453a0 0x7f1080145810 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f10800b9ab0 tx=0x7f107801c490 comp rx=0 tx=0).stop 2024-03-18T15:34:20.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.892+0000 7f10882cc640 1 -- 172.21.15.5:0/580091926 shutdown_connections 2024-03-18T15:34:20.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.892+0000 7f10882cc640 1 --2- 172.21.15.5:0/580091926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10801453a0 0x7f1080145810 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:20.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.892+0000 7f10882cc640 1 --2- 172.21.15.5:0/580091926 >> v2:172.21.15.80:3300/0 conn(0x7f108013f260 0x7f108013f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:20.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.892+0000 7f10882cc640 1 --2- 172.21.15.5:0/580091926 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f108006d2d0 0x7f10800b23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:20.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.892+0000 7f10882cc640 1 -- 172.21.15.5:0/580091926 >> 172.21.15.5:0/580091926 conn(0x7f10800b5670 msgr2=0x7f10800b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:20.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.892+0000 7f10882cc640 1 -- 172.21.15.5:0/580091926 shutdown_connections 2024-03-18T15:34:20.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.892+0000 7f10882cc640 1 -- 172.21.15.5:0/580091926 wait complete. 2024-03-18T15:34:20.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.896+0000 7f10882cc640 1 Processor -- start 2024-03-18T15:34:20.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.896+0000 7f10882cc640 1 -- start start 2024-03-18T15:34:20.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.896+0000 7f10882cc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f108006d2d0 0x7f10801484f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:20.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.896+0000 7f10882cc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f108013f260 0x7f10801dfe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:20.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.896+0000 7f10882cc640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f10801453a0 0x7f10801e0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:20.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.896+0000 7f10882cc640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f10800b0960 con 0x7f108006d2d0 2024-03-18T15:34:20.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.896+0000 7f10882cc640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f10800b07e0 con 0x7f108013f260 2024-03-18T15:34:20.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.896+0000 7f10882cc640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f10800b5140 con 0x7f10801453a0 2024-03-18T15:34:20.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.896+0000 7f1086068640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f108006d2d0 0x7f10801484f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:20.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.896+0000 7f1086068640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f108006d2d0 0x7f10801484f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44952/0 (socket says 172.21.15.5:44952) 2024-03-18T15:34:20.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.896+0000 7f1086068640 1 -- 172.21.15.5:0/149234785 learned_addr learned my addr 172.21.15.5:0/149234785 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:20.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.896+0000 7f1085867640 1 --2- 172.21.15.5:0/149234785 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f108013f260 0x7f10801dfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:20.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.896+0000 7f1086869640 1 --2- 172.21.15.5:0/149234785 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f10801453a0 0x7f10801e0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:20.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.896+0000 7f1086068640 1 -- 172.21.15.5:0/149234785 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f10801453a0 msgr2=0x7f10801e0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:20.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.896+0000 7f1086068640 1 --2- 172.21.15.5:0/149234785 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f10801453a0 0x7f10801e0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:20.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.896+0000 7f1086068640 1 -- 172.21.15.5:0/149234785 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f108013f260 msgr2=0x7f10801dfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:20.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.896+0000 7f1086068640 1 --2- 172.21.15.5:0/149234785 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f108013f260 0x7f10801dfe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:20.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.896+0000 7f1086068640 1 -- 172.21.15.5:0/149234785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1080139520 con 0x7f108006d2d0 2024-03-18T15:34:20.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.896+0000 7f1086068640 1 --2- 172.21.15.5:0/149234785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f108006d2d0 0x7f10801484f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f107c00c7f0 tx=0x7f107c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:20.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.896+0000 7f1072ffd640 1 -- 172.21.15.5:0/149234785 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f107c015070 con 0x7f108006d2d0 2024-03-18T15:34:20.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.896+0000 7f1072ffd640 1 -- 172.21.15.5:0/149234785 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f107c004220 con 0x7f108006d2d0 2024-03-18T15:34:20.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.896+0000 7f1072ffd640 1 -- 172.21.15.5:0/149234785 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f107c003620 con 0x7f108006d2d0 2024-03-18T15:34:20.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.896+0000 7f10882cc640 1 -- 172.21.15.5:0/149234785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1080139240 con 0x7f108006d2d0 2024-03-18T15:34:20.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.896+0000 7f10882cc640 1 -- 172.21.15.5:0/149234785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10801475b0 con 0x7f108006d2d0 2024-03-18T15:34:20.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.896+0000 7f10882cc640 1 -- 172.21.15.5:0/149234785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f108004f550 con 0x7f108006d2d0 2024-03-18T15:34:20.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.896+0000 7f1072ffd640 1 -- 172.21.15.5:0/149234785 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f107c0039d0 con 0x7f108006d2d0 2024-03-18T15:34:20.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.900+0000 7f1072ffd640 1 --2- 172.21.15.5:0/149234785 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f1064067660 0x7f1064069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:20.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.900+0000 7f1072ffd640 1 -- 172.21.15.5:0/149234785 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5227+0+0 (secure 0 0 0) 0x7f107c085af0 con 0x7f108006d2d0 2024-03-18T15:34:20.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.900+0000 7f1085867640 1 --2- 172.21.15.5:0/149234785 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f1064067660 0x7f1064069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:20.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.900+0000 7f1085867640 1 --2- 172.21.15.5:0/149234785 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f1064067660 0x7f1064069b20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f10740075b0 tx=0x7f1074007540 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:20.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:20.904+0000 7f1072ffd640 1 -- 172.21.15.5:0/149234785 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f1080139520 con 0x7f108006d2d0 2024-03-18T15:34:21.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.064+0000 7f10882cc640 1 -- 172.21.15.5:0/149234785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f108014dd60 con 0x7f108006d2d0 2024-03-18T15:34:21.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.728+0000 7f1072ffd640 1 -- 172.21.15.5:0/149234785 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v28) ==== 170+0+0 (secure 0 0 0) 0x7f107c056a30 con 0x7f108006d2d0 2024-03-18T15:34:21.733 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' created 2024-03-18T15:34:21.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.732+0000 7f10882cc640 1 -- 172.21.15.5:0/149234785 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f1064067660 msgr2=0x7f1064069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:21.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.732+0000 7f10882cc640 1 --2- 172.21.15.5:0/149234785 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f1064067660 0x7f1064069b20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f10740075b0 tx=0x7f1074007540 comp rx=0 tx=0).stop 2024-03-18T15:34:21.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.732+0000 7f10882cc640 1 -- 172.21.15.5:0/149234785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f108006d2d0 msgr2=0x7f10801484f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:21.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.732+0000 7f10882cc640 1 --2- 172.21.15.5:0/149234785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f108006d2d0 0x7f10801484f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f107c00c7f0 tx=0x7f107c00ccb0 comp rx=0 tx=0).stop 2024-03-18T15:34:21.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.732+0000 7f10882cc640 1 -- 172.21.15.5:0/149234785 shutdown_connections 2024-03-18T15:34:21.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.732+0000 7f10882cc640 1 --2- 172.21.15.5:0/149234785 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f1064067660 0x7f1064069b20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:21.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.732+0000 7f10882cc640 1 --2- 172.21.15.5:0/149234785 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f10801453a0 0x7f10801e0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:21.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.732+0000 7f10882cc640 1 --2- 172.21.15.5:0/149234785 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f108013f260 0x7f10801dfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:21.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.732+0000 7f10882cc640 1 --2- 172.21.15.5:0/149234785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f108006d2d0 0x7f10801484f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:21.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.732+0000 7f10882cc640 1 -- 172.21.15.5:0/149234785 >> 172.21.15.5:0/149234785 conn(0x7f10800b5670 msgr2=0x7f108014f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:21.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.732+0000 7f10882cc640 1 -- 172.21.15.5:0/149234785 shutdown_connections 2024-03-18T15:34:21.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.732+0000 7f10882cc640 1 -- 172.21.15.5:0/149234785 wait complete. 2024-03-18T15:34:21.771 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-18T15:34:21.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.892+0000 7fcadf671640 1 Processor -- start 2024-03-18T15:34:21.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.892+0000 7fcadf671640 1 -- start start 2024-03-18T15:34:21.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.892+0000 7fcadf671640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcad806d2d0 0x7fcad80b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:21.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.892+0000 7fcadf671640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcad813f260 0x7fcad813f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:21.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.892+0000 7fcadf671640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcad80b0ea0 con 0x7fcad81453a0 2024-03-18T15:34:21.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.892+0000 7fcadf671640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fcad80728c0 con 0x7fcad806d2d0 2024-03-18T15:34:21.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.892+0000 7fcadf671640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fcad806d6b0 con 0x7fcad813f260 2024-03-18T15:34:21.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.892+0000 7fcadd40d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fcad806d2d0 0x7fcad80b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:21.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.892+0000 7fcaddc0e640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fcad81453a0 0x7fcad8145810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47008/0 (socket says 172.21.15.5:47008) 2024-03-18T15:34:21.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.892+0000 7fcaddc0e640 1 -- 172.21.15.5:0/3848319586 learned_addr learned my addr 172.21.15.5:0/3848319586 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:21.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.892+0000 7fcadcc0c640 1 --2- 172.21.15.5:0/3848319586 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcad813f260 0x7fcad813f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:21.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.892+0000 7fcadd40d640 1 -- 172.21.15.5:0/3848319586 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcad813f260 msgr2=0x7fcad813f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:21.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.892+0000 7fcadd40d640 1 --2- 172.21.15.5:0/3848319586 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcad813f260 0x7fcad813f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:21.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.892+0000 7fcadd40d640 1 -- 172.21.15.5:0/3848319586 >> v1:172.21.15.5:6789/0 conn(0x7fcad81453a0 legacy=0x7fcad8145810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:21.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.892+0000 7fcadd40d640 1 -- 172.21.15.5:0/3848319586 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad80b0bc0 con 0x7fcad806d2d0 2024-03-18T15:34:21.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.892+0000 7fcad7fff640 1 -- 172.21.15.5:0/3848319586 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1699704137 0 0) 0x7fcad80b0ea0 con 0x7fcad81453a0 2024-03-18T15:34:21.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.892+0000 7fcadcc0c640 1 --2- 172.21.15.5:0/3848319586 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcad813f260 0x7fcad813f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:21.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.892+0000 7fcadd40d640 1 --2- 172.21.15.5:0/3848319586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcad806d2d0 0x7fcad80b23a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fcad0009d90 tx=0x7fcad001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f163083973f5cc3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:21.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.892+0000 7fcad7fff640 1 -- 172.21.15.5:0/3848319586 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad00b2070 con 0x7fcad806d2d0 2024-03-18T15:34:21.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.896+0000 7fcad7fff640 1 -- 172.21.15.5:0/3848319586 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcad00a58b0 con 0x7fcad806d2d0 2024-03-18T15:34:21.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.896+0000 7fcad7fff640 1 -- 172.21.15.5:0/3848319586 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad00a5c80 con 0x7fcad806d2d0 2024-03-18T15:34:21.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.896+0000 7fcadf671640 1 -- 172.21.15.5:0/3848319586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcad806d2d0 msgr2=0x7fcad80b23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:21.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.896+0000 7fcadf671640 1 --2- 172.21.15.5:0/3848319586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcad806d2d0 0x7fcad80b23a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fcad0009d90 tx=0x7fcad001c8e0 comp rx=0 tx=0).stop 2024-03-18T15:34:21.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.896+0000 7fcadf671640 1 -- 172.21.15.5:0/3848319586 shutdown_connections 2024-03-18T15:34:21.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.896+0000 7fcadf671640 1 --2- 172.21.15.5:0/3848319586 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcad813f260 0x7fcad813f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:21.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.896+0000 7fcadf671640 1 --2- 172.21.15.5:0/3848319586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcad806d2d0 0x7fcad80b23a0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:21.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.896+0000 7fcadf671640 1 -- 172.21.15.5:0/3848319586 >> 172.21.15.5:0/3848319586 conn(0x7fcad80b5670 msgr2=0x7fcad80b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:21.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.896+0000 7fcadf671640 1 -- 172.21.15.5:0/3848319586 shutdown_connections 2024-03-18T15:34:21.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.896+0000 7fcadf671640 1 -- 172.21.15.5:0/3848319586 wait complete. 2024-03-18T15:34:21.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.896+0000 7fcadf671640 1 Processor -- start 2024-03-18T15:34:21.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.896+0000 7fcadf671640 1 -- start start 2024-03-18T15:34:21.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.896+0000 7fcadf671640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcad806d2d0 0x7fcad8149040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:21.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.896+0000 7fcadf671640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcad813f260 0x7fcad813ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:21.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.896+0000 7fcadf671640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcad81453a0 0x7fcad81404f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:21.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.896+0000 7fcadf671640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcad8072ed0 con 0x7fcad806d2d0 2024-03-18T15:34:21.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.896+0000 7fcadf671640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fcad8072d50 con 0x7fcad81453a0 2024-03-18T15:34:21.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.896+0000 7fcadf671640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fcad80b5140 con 0x7fcad813f260 2024-03-18T15:34:21.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.896+0000 7fcadd40d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcad806d2d0 0x7fcad8149040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:21.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.896+0000 7fcadd40d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcad806d2d0 0x7fcad8149040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44970/0 (socket says 172.21.15.5:44970) 2024-03-18T15:34:21.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.896+0000 7fcadd40d640 1 -- 172.21.15.5:0/3482623666 learned_addr learned my addr 172.21.15.5:0/3482623666 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:21.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.896+0000 7fcaddc0e640 1 --2- 172.21.15.5:0/3482623666 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcad81453a0 0x7fcad81404f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:21.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.896+0000 7fcadd40d640 1 -- 172.21.15.5:0/3482623666 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcad813f260 msgr2=0x7fcad813ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:21.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.896+0000 7fcadcc0c640 1 --2- 172.21.15.5:0/3482623666 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcad813f260 0x7fcad813ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:21.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.896+0000 7fcadd40d640 1 --2- 172.21.15.5:0/3482623666 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcad813f260 0x7fcad813ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:21.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.896+0000 7fcadd40d640 1 -- 172.21.15.5:0/3482623666 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcad81453a0 msgr2=0x7fcad81404f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:21.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.896+0000 7fcadd40d640 1 --2- 172.21.15.5:0/3482623666 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcad81453a0 0x7fcad81404f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:21.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.896+0000 7fcadd40d640 1 -- 172.21.15.5:0/3482623666 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad8142a90 con 0x7fcad806d2d0 2024-03-18T15:34:21.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.896+0000 7fcaddc0e640 1 --2- 172.21.15.5:0/3482623666 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcad81453a0 0x7fcad81404f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:21.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.896+0000 7fcadcc0c640 1 --2- 172.21.15.5:0/3482623666 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcad813f260 0x7fcad813ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:21.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.896+0000 7fcadd40d640 1 --2- 172.21.15.5:0/3482623666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcad806d2d0 0x7fcad8149040 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fcad0009d90 tx=0x7fcad0019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:21.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.896+0000 7fcad5ffb640 1 -- 172.21.15.5:0/3482623666 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad00b2050 con 0x7fcad806d2d0 2024-03-18T15:34:21.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.896+0000 7fcad5ffb640 1 -- 172.21.15.5:0/3482623666 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcad00b3040 con 0x7fcad806d2d0 2024-03-18T15:34:21.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.896+0000 7fcad5ffb640 1 -- 172.21.15.5:0/3482623666 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad00ad850 con 0x7fcad806d2d0 2024-03-18T15:34:21.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.896+0000 7fcadf671640 1 -- 172.21.15.5:0/3482623666 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcad81418e0 con 0x7fcad806d2d0 2024-03-18T15:34:21.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.896+0000 7fcadf671640 1 -- 172.21.15.5:0/3482623666 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcad8140c70 con 0x7fcad806d2d0 2024-03-18T15:34:21.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.900+0000 7fcadf671640 1 -- 172.21.15.5:0/3482623666 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcad804f550 con 0x7fcad806d2d0 2024-03-18T15:34:21.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.900+0000 7fcad5ffb640 1 -- 172.21.15.5:0/3482623666 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fcad00c0020 con 0x7fcad806d2d0 2024-03-18T15:34:21.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.900+0000 7fcad5ffb640 1 --2- 172.21.15.5:0/3482623666 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fcac4067660 0x7fcac4069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:21.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.900+0000 7fcad5ffb640 1 -- 172.21.15.5:0/3482623666 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 5609+0+0 (secure 0 0 0) 0x7fcad01200e0 con 0x7fcad806d2d0 2024-03-18T15:34:21.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.900+0000 7fcadcc0c640 1 --2- 172.21.15.5:0/3482623666 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fcac4067660 0x7fcac4069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:21.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.900+0000 7fcadcc0c640 1 --2- 172.21.15.5:0/3482623666 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fcac4067660 0x7fcac4069b20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcad80b6ae0 tx=0x7fcacc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:21.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:21.904+0000 7fcad5ffb640 1 -- 172.21.15.5:0/3482623666 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fcad0016560 con 0x7fcad806d2d0 2024-03-18T15:34:22.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.104+0000 7fcadf671640 1 -- 172.21.15.5:0/3482623666 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fcad81434b0 con 0x7fcad806d2d0 2024-03-18T15:34:22.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.732+0000 7fcad5ffb640 1 -- 172.21.15.5:0/3482623666 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) ==== 172+0+0 (secure 0 0 0) 0x7fcad00f0e50 con 0x7fcad806d2d0 2024-03-18T15:34:22.736 INFO:teuthology.orchestra.run.smithi005.stderr:new fs with metadata pool 4 and data pool 5 2024-03-18T15:34:22.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.736+0000 7fcadf671640 1 -- 172.21.15.5:0/3482623666 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fcac4067660 msgr2=0x7fcac4069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:22.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.736+0000 7fcadf671640 1 --2- 172.21.15.5:0/3482623666 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fcac4067660 0x7fcac4069b20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcad80b6ae0 tx=0x7fcacc008040 comp rx=0 tx=0).stop 2024-03-18T15:34:22.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.736+0000 7fcadf671640 1 -- 172.21.15.5:0/3482623666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcad806d2d0 msgr2=0x7fcad8149040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:22.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.736+0000 7fcadf671640 1 --2- 172.21.15.5:0/3482623666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcad806d2d0 0x7fcad8149040 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fcad0009d90 tx=0x7fcad0019680 comp rx=0 tx=0).stop 2024-03-18T15:34:22.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.736+0000 7fcadf671640 1 -- 172.21.15.5:0/3482623666 shutdown_connections 2024-03-18T15:34:22.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.740+0000 7fcadf671640 1 --2- 172.21.15.5:0/3482623666 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fcac4067660 0x7fcac4069b20 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:22.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.740+0000 7fcadf671640 1 --2- 172.21.15.5:0/3482623666 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcad81453a0 0x7fcad81404f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:22.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.740+0000 7fcadf671640 1 --2- 172.21.15.5:0/3482623666 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fcad813f260 0x7fcad813ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:22.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.740+0000 7fcadf671640 1 --2- 172.21.15.5:0/3482623666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcad806d2d0 0x7fcad8149040 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:22.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.740+0000 7fcadf671640 1 -- 172.21.15.5:0/3482623666 >> 172.21.15.5:0/3482623666 conn(0x7fcad80b5670 msgr2=0x7fcad80b5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:22.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.740+0000 7fcadf671640 1 -- 172.21.15.5:0/3482623666 shutdown_connections 2024-03-18T15:34:22.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.740+0000 7fcadf671640 1 -- 172.21.15.5:0/3482623666 wait complete. 2024-03-18T15:34:22.786 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:34:22.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.904+0000 7f8cd81b1640 1 Processor -- start 2024-03-18T15:34:22.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.904+0000 7f8cd81b1640 1 -- start start 2024-03-18T15:34:22.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.904+0000 7f8cd81b1640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8cd0144c20 0x7f8cd0140fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:22.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.904+0000 7f8cd81b1640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8cd00b5450 0x7f8cd0141510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:22.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.904+0000 7f8cd81b1640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8cd0073530 con 0x7f8cd00744b0 2024-03-18T15:34:22.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.904+0000 7f8cd81b1640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f8cd00b34a0 con 0x7f8cd00b5450 2024-03-18T15:34:22.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.904+0000 7f8cd81b1640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f8cd00b27a0 con 0x7f8cd0144c20 2024-03-18T15:34:22.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.904+0000 7f8cd674e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8cd00b5450 0x7f8cd0141510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:22.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.904+0000 7f8cd674e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8cd00b5450 0x7f8cd0141510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44984/0 (socket says 172.21.15.5:44984) 2024-03-18T15:34:22.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.904+0000 7f8cd674e640 1 -- 172.21.15.5:0/104908839 learned_addr learned my addr 172.21.15.5:0/104908839 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:22.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.904+0000 7f8cd574c640 1 --2- 172.21.15.5:0/104908839 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8cd0144c20 0x7f8cd0140fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:22.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.908+0000 7f8cd674e640 1 -- 172.21.15.5:0/104908839 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8cd0144c20 msgr2=0x7f8cd0140fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:22.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.908+0000 7f8cd674e640 1 --2- 172.21.15.5:0/104908839 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8cd0144c20 0x7f8cd0140fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:22.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.908+0000 7f8cd674e640 1 -- 172.21.15.5:0/104908839 >> v1:172.21.15.80:6789/0 conn(0x7f8cd00744b0 legacy=0x7f8cd0144510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:22.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.908+0000 7f8cd674e640 1 -- 172.21.15.5:0/104908839 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cd0074890 con 0x7f8cd00b5450 2024-03-18T15:34:22.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.908+0000 7f8cd574c640 1 --2- 172.21.15.5:0/104908839 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8cd0144c20 0x7f8cd0140fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:22.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.908+0000 7f8cd674e640 1 --2- 172.21.15.5:0/104908839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cd00b5450 0x7f8cd0141510 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8cc800d150 tx=0x7f8cc801eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d14042ca5eda215 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:22.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.908+0000 7f8cd4f4b640 1 -- 172.21.15.5:0/104908839 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc800b2d0 con 0x7f8cd00b5450 2024-03-18T15:34:22.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.908+0000 7f8cd4f4b640 1 -- 172.21.15.5:0/104908839 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cc800b470 con 0x7f8cd00b5450 2024-03-18T15:34:22.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.908+0000 7f8cd4f4b640 1 -- 172.21.15.5:0/104908839 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc80a7a70 con 0x7f8cd00b5450 2024-03-18T15:34:22.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.908+0000 7f8cd81b1640 1 -- 172.21.15.5:0/104908839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cd00b5450 msgr2=0x7f8cd0141510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:22.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.908+0000 7f8cd81b1640 1 --2- 172.21.15.5:0/104908839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cd00b5450 0x7f8cd0141510 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8cc800d150 tx=0x7f8cc801eb70 comp rx=0 tx=0).stop 2024-03-18T15:34:22.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.908+0000 7f8cd81b1640 1 -- 172.21.15.5:0/104908839 shutdown_connections 2024-03-18T15:34:22.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.908+0000 7f8cd81b1640 1 --2- 172.21.15.5:0/104908839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cd00b5450 0x7f8cd0141510 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:22.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.908+0000 7f8cd81b1640 1 --2- 172.21.15.5:0/104908839 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8cd0144c20 0x7f8cd0140fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:22.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.908+0000 7f8cd81b1640 1 -- 172.21.15.5:0/104908839 >> 172.21.15.5:0/104908839 conn(0x7f8cd00b9190 msgr2=0x7f8cd00b95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:22.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.908+0000 7f8cd81b1640 1 -- 172.21.15.5:0/104908839 shutdown_connections 2024-03-18T15:34:22.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.908+0000 7f8cd81b1640 1 -- 172.21.15.5:0/104908839 wait complete. 2024-03-18T15:34:22.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.912+0000 7f8cd81b1640 1 Processor -- start 2024-03-18T15:34:22.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.912+0000 7f8cd81b1640 1 -- start start 2024-03-18T15:34:22.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.912+0000 7f8cd81b1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8cd00744b0 0x7f8cd0140800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:22.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.912+0000 7f8cd81b1640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8cd00b5450 0x7f8cd00ac7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:22.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.912+0000 7f8cd81b1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cd0144c20 0x7f8cd00a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:22.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.912+0000 7f8cd81b1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd00b4320 con 0x7f8cd0144c20 2024-03-18T15:34:22.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.912+0000 7f8cd81b1640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd00b2d30 con 0x7f8cd00744b0 2024-03-18T15:34:22.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.912+0000 7f8cd81b1640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f8cd00b44a0 con 0x7f8cd00b5450 2024-03-18T15:34:22.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.912+0000 7f8cd674e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cd0144c20 0x7f8cd00a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:22.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.912+0000 7f8cd674e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cd0144c20 0x7f8cd00a94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44986/0 (socket says 172.21.15.5:44986) 2024-03-18T15:34:22.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.912+0000 7f8cd674e640 1 -- 172.21.15.5:0/677907457 learned_addr learned my addr 172.21.15.5:0/677907457 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:22.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.912+0000 7f8cd574c640 1 --2- 172.21.15.5:0/677907457 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8cd00b5450 0x7f8cd00ac7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:22.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.912+0000 7f8cd5f4d640 1 --2- 172.21.15.5:0/677907457 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8cd00744b0 0x7f8cd0140800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:22.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.912+0000 7f8cd674e640 1 -- 172.21.15.5:0/677907457 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8cd00b5450 msgr2=0x7f8cd00ac7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:22.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.912+0000 7f8cd674e640 1 --2- 172.21.15.5:0/677907457 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8cd00b5450 0x7f8cd00ac7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:22.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.912+0000 7f8cd674e640 1 -- 172.21.15.5:0/677907457 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8cd00744b0 msgr2=0x7f8cd0140800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:22.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.912+0000 7f8cd674e640 1 --2- 172.21.15.5:0/677907457 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8cd00744b0 0x7f8cd0140800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:22.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.912+0000 7f8cd674e640 1 -- 172.21.15.5:0/677907457 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cd00add60 con 0x7f8cd0144c20 2024-03-18T15:34:22.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.912+0000 7f8cd574c640 1 --2- 172.21.15.5:0/677907457 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8cd00b5450 0x7f8cd00ac7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:22.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.912+0000 7f8cd5f4d640 1 --2- 172.21.15.5:0/677907457 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8cd00744b0 0x7f8cd0140800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:22.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.912+0000 7f8cd674e640 1 --2- 172.21.15.5:0/677907457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cd0144c20 0x7f8cd00a94d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8cc801c7a0 tx=0x7f8cc801b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:22.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.912+0000 7f8cc67fc640 1 -- 172.21.15.5:0/677907457 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc80be020 con 0x7f8cd0144c20 2024-03-18T15:34:22.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.912+0000 7f8cc67fc640 1 -- 172.21.15.5:0/677907457 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8cc8009820 con 0x7f8cd0144c20 2024-03-18T15:34:22.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.912+0000 7f8cd81b1640 1 -- 172.21.15.5:0/677907457 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cd00aeb10 con 0x7f8cd0144c20 2024-03-18T15:34:22.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.912+0000 7f8cc67fc640 1 -- 172.21.15.5:0/677907457 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc8009c10 con 0x7f8cd0144c20 2024-03-18T15:34:22.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.912+0000 7f8cd81b1640 1 -- 172.21.15.5:0/677907457 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cd00ab1b0 con 0x7f8cd0144c20 2024-03-18T15:34:22.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.912+0000 7f8cd81b1640 1 -- 172.21.15.5:0/677907457 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cd00ac4d0 con 0x7f8cd0144c20 2024-03-18T15:34:22.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.912+0000 7f8cc67fc640 1 -- 172.21.15.5:0/677907457 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f8cc80c1020 con 0x7f8cd0144c20 2024-03-18T15:34:22.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.912+0000 7f8cc67fc640 1 --2- 172.21.15.5:0/677907457 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f8cb4067660 0x7f8cb4069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:22.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.916+0000 7f8cc67fc640 1 -- 172.21.15.5:0/677907457 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f8cc8123500 con 0x7f8cd0144c20 2024-03-18T15:34:22.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.916+0000 7f8cd5f4d640 1 --2- 172.21.15.5:0/677907457 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f8cb4067660 0x7f8cb4069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:22.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.916+0000 7f8cd5f4d640 1 --2- 172.21.15.5:0/677907457 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f8cb4067660 0x7f8cb4069b20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc0016f0 tx=0x7f8ccc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:22.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:22.916+0000 7f8cc67fc640 1 -- 172.21.15.5:0/677907457 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f8cd00ac4d0 con 0x7f8cd0144c20 2024-03-18T15:34:23.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.080+0000 7f8cd81b1640 1 -- 172.21.15.5:0/677907457 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8cd0072310 con 0x7f8cd0144c20 2024-03-18T15:34:23.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.080+0000 7f8cc67fc640 1 -- 172.21.15.5:0/677907457 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+16526 (secure 0 0 0) 0x7f8cc8018820 con 0x7f8cd0144c20 2024-03-18T15:34:23.083 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:34:23.083 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":29,"fsid":"ef54fe03-5aa7-4eb2-9b1e-66c9cc345b94","created":"2024-03-18T15:33:18.086614+0000","modified":"2024-03-18T15:34:22.107767+0000","last_up_change":"2024-03-18T15:33:29.229911+0000","last_in_change":"2024-03-18T15:33:20.323840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:28.963186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-18T15:34:20.652674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-18T15:34:21.110594+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df1c754b-af59-4d16-8919-fbb6386b3c48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6804","nonce":976264754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6808","nonce":976264754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6815","nonce":976264754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6811","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6813","nonce":976264754}]},"public_addr":"172.21.15.5:6804/976264754","cluster_addr":"172.21.15.5:6808/976264754","heartbeat_back_addr":"172.21.15.5:6815/976264754","heartbeat_front_addr":"172.21.15.5:6813/976264754","state":["exists","up"]},{"osd":1,"uuid":"b9a08b5e-8af4-4144-bdba-d69ed1ac627a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6801","nonce":3901761126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6803","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6805","nonce":3901761126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6812","nonce":3901761126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6807","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6809","nonce":3901761126}]},"public_addr":"172.21.15.5:6801/3901761126","cluster_addr":"172.21.15.5:6805/3901761126","heartbeat_back_addr":"172.21.15.5:6812/3901761126","heartbeat_front_addr":"172.21.15.5:6809/3901761126","state":["exists","up"]},{"osd":2,"uuid":"c3e8bb76-e382-439d-b380-62ef92f1255a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6817","nonce":502968448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6819","nonce":502968448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6823","nonce":502968448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6821","nonce":502968448}]},"public_addr":"172.21.15.5:6817/502968448","cluster_addr":"172.21.15.5:6819/502968448","heartbeat_back_addr":"172.21.15.5:6823/502968448","heartbeat_front_addr":"172.21.15.5:6821/502968448","state":["exists","up"]},{"osd":3,"uuid":"e8c94625-f2cd-4eba-b53d-2e7356e1c02a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6825","nonce":2425932735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6827","nonce":2425932735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6831","nonce":2425932735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6829","nonce":2425932735}]},"public_addr":"172.21.15.5:6825/2425932735","cluster_addr":"172.21.15.5:6827/2425932735","heartbeat_back_addr":"172.21.15.5:6831/2425932735","heartbeat_front_addr":"172.21.15.5:6829/2425932735","state":["exists","up"]},{"osd":4,"uuid":"0099dd52-6265-4d32-82e4-3ef16e73add0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6817","nonce":976514285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6819","nonce":976514285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6823","nonce":976514285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6821","nonce":976514285}]},"public_addr":"172.21.15.80:6817/976514285","cluster_addr":"172.21.15.80:6819/976514285","heartbeat_back_addr":"172.21.15.80:6823/976514285","heartbeat_front_addr":"172.21.15.80:6821/976514285","state":["exists","up"]},{"osd":5,"uuid":"ed10c84f-8c39-41cc-9b64-7c482a94a48e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6825","nonce":627153447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6827","nonce":627153447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6831","nonce":627153447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6829","nonce":627153447}]},"public_addr":"172.21.15.80:6825/627153447","cluster_addr":"172.21.15.80:6827/627153447","heartbeat_back_addr":"172.21.15.80:6831/627153447","heartbeat_front_addr":"172.21.15.80:6829/627153447","state":["exists","up"]},{"osd":6,"uuid":"c7a3538a-7048-4fba-8781-b440f03c25db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6803","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6805","nonce":488342527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6807","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6809","nonce":488342527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6815","nonce":488342527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6811","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6813","nonce":488342527}]},"public_addr":"172.21.15.80:6805/488342527","cluster_addr":"172.21.15.80:6809/488342527","heartbeat_back_addr":"172.21.15.80:6815/488342527","heartbeat_front_addr":"172.21.15.80:6813/488342527","state":["exists","up"]},{"osd":7,"uuid":"f9418104-3a97-4110-bc3a-1f8151d0f757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6801","nonce":982831666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6804","nonce":982831666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6812","nonce":982831666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6808","nonce":982831666}]},"public_addr":"172.21.15.80:6801/982831666","cluster_addr":"172.21.15.80:6804/982831666","heartbeat_back_addr":"172.21.15.80:6812/982831666","heartbeat_front_addr":"172.21.15.80:6808/982831666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.970828+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.858116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.987556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.864603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.756293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.591502+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:34:23.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.080+0000 7f8cd81b1640 1 -- 172.21.15.5:0/677907457 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f8cb4067660 msgr2=0x7f8cb4069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:23.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.080+0000 7f8cd81b1640 1 --2- 172.21.15.5:0/677907457 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f8cb4067660 0x7f8cb4069b20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc0016f0 tx=0x7f8ccc009040 comp rx=0 tx=0).stop 2024-03-18T15:34:23.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.080+0000 7f8cd81b1640 1 -- 172.21.15.5:0/677907457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cd0144c20 msgr2=0x7f8cd00a94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:23.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.080+0000 7f8cd81b1640 1 --2- 172.21.15.5:0/677907457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cd0144c20 0x7f8cd00a94d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f8cc801c7a0 tx=0x7f8cc801b910 comp rx=0 tx=0).stop 2024-03-18T15:34:23.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.080+0000 7f8cd81b1640 1 -- 172.21.15.5:0/677907457 shutdown_connections 2024-03-18T15:34:23.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.080+0000 7f8cd81b1640 1 --2- 172.21.15.5:0/677907457 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f8cb4067660 0x7f8cb4069b20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:23.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.080+0000 7f8cd81b1640 1 --2- 172.21.15.5:0/677907457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cd0144c20 0x7f8cd00a94d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:23.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.080+0000 7f8cd81b1640 1 --2- 172.21.15.5:0/677907457 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8cd00b5450 0x7f8cd00ac7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:23.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.080+0000 7f8cd81b1640 1 --2- 172.21.15.5:0/677907457 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8cd00744b0 0x7f8cd0140800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:23.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.080+0000 7f8cd81b1640 1 -- 172.21.15.5:0/677907457 >> 172.21.15.5:0/677907457 conn(0x7f8cd00b9190 msgr2=0x7f8cd00b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:23.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.080+0000 7f8cd81b1640 1 -- 172.21.15.5:0/677907457 shutdown_connections 2024-03-18T15:34:23.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.084+0000 7f8cd81b1640 1 -- 172.21.15.5:0/677907457 wait complete. 2024-03-18T15:34:23.101 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:34:23.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.224+0000 7fd3f222f640 1 Processor -- start 2024-03-18T15:34:23.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.228+0000 7fd3f222f640 1 -- start start 2024-03-18T15:34:23.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.228+0000 7fd3f222f640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd3ec0744b0 0x7fd3ec144510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:23.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.228+0000 7fd3f222f640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd3ec144a50 0x7fd3ec147f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:23.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.228+0000 7fd3f222f640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3ec144ee0 con 0x7fd3ec0b5450 2024-03-18T15:34:23.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.228+0000 7fd3f222f640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fd3ec0b34a0 con 0x7fd3ec0744b0 2024-03-18T15:34:23.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.228+0000 7fd3f222f640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fd3ec0b27a0 con 0x7fd3ec144a50 2024-03-18T15:34:23.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.228+0000 7fd3eb7fe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd3ec0744b0 0x7fd3ec144510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:23.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.228+0000 7fd3eaffd640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd3ec0b5450 0x7fd3ec1477f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47014/0 (socket says 172.21.15.5:47014) 2024-03-18T15:34:23.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.228+0000 7fd3eaffd640 1 -- 172.21.15.5:0/1238162640 learned_addr learned my addr 172.21.15.5:0/1238162640 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:23.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.228+0000 7fd3ebfff640 1 --2- 172.21.15.5:0/1238162640 >> v2:172.21.15.80:3300/0 conn(0x7fd3ec144a50 0x7fd3ec147f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:23.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.228+0000 7fd3eb7fe640 1 -- 172.21.15.5:0/1238162640 >> v2:172.21.15.80:3300/0 conn(0x7fd3ec144a50 msgr2=0x7fd3ec147f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:23.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.228+0000 7fd3eb7fe640 1 --2- 172.21.15.5:0/1238162640 >> v2:172.21.15.80:3300/0 conn(0x7fd3ec144a50 0x7fd3ec147f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:23.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.228+0000 7fd3eb7fe640 1 -- 172.21.15.5:0/1238162640 >> v1:172.21.15.5:6789/0 conn(0x7fd3ec0b5450 legacy=0x7fd3ec1477f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:23.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.228+0000 7fd3eb7fe640 1 -- 172.21.15.5:0/1238162640 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3ec073530 con 0x7fd3ec0744b0 2024-03-18T15:34:23.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.228+0000 7fd3eb7fe640 1 --2- 172.21.15.5:0/1238162640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3ec0744b0 0x7fd3ec144510 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec0b0200 tx=0x7fd3dc01c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dc692c1d882f4e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:23.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.232+0000 7fd3ea7fc640 1 -- 172.21.15.5:0/1238162640 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc0b2070 con 0x7fd3ec0744b0 2024-03-18T15:34:23.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.232+0000 7fd3ea7fc640 1 -- 172.21.15.5:0/1238162640 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3dc0a9070 con 0x7fd3ec0744b0 2024-03-18T15:34:23.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.232+0000 7fd3ea7fc640 1 -- 172.21.15.5:0/1238162640 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc0ad880 con 0x7fd3ec0744b0 2024-03-18T15:34:23.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.232+0000 7fd3f222f640 1 -- 172.21.15.5:0/1238162640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3ec0744b0 msgr2=0x7fd3ec144510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:23.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.232+0000 7fd3f222f640 1 --2- 172.21.15.5:0/1238162640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3ec0744b0 0x7fd3ec144510 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec0b0200 tx=0x7fd3dc01c360 comp rx=0 tx=0).stop 2024-03-18T15:34:23.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.232+0000 7fd3f222f640 1 -- 172.21.15.5:0/1238162640 shutdown_connections 2024-03-18T15:34:23.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.232+0000 7fd3f222f640 1 --2- 172.21.15.5:0/1238162640 >> v2:172.21.15.80:3300/0 conn(0x7fd3ec144a50 0x7fd3ec147f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:23.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.232+0000 7fd3f222f640 1 --2- 172.21.15.5:0/1238162640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3ec0744b0 0x7fd3ec144510 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:23.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.232+0000 7fd3f222f640 1 -- 172.21.15.5:0/1238162640 >> 172.21.15.5:0/1238162640 conn(0x7fd3ec0b9190 msgr2=0x7fd3ec0b95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:23.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.232+0000 7fd3f222f640 1 -- 172.21.15.5:0/1238162640 shutdown_connections 2024-03-18T15:34:23.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.232+0000 7fd3f222f640 1 -- 172.21.15.5:0/1238162640 wait complete. 2024-03-18T15:34:23.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.232+0000 7fd3f222f640 1 Processor -- start 2024-03-18T15:34:23.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.232+0000 7fd3f222f640 1 -- start start 2024-03-18T15:34:23.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.232+0000 7fd3f222f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3ec0744b0 0x7fd3ec140e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:23.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.232+0000 7fd3f222f640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd3ec0b5450 0x7fd3ec13c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:23.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.232+0000 7fd3f222f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd3ec144a50 0x7fd3ec143c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:23.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.232+0000 7fd3f222f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ec06fb50 con 0x7fd3ec0744b0 2024-03-18T15:34:23.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.232+0000 7fd3f222f640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ec0b2ab0 con 0x7fd3ec144a50 2024-03-18T15:34:23.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.232+0000 7fd3f222f640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fd3ec148440 con 0x7fd3ec0b5450 2024-03-18T15:34:23.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.232+0000 7fd3eb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3ec0744b0 0x7fd3ec140e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:23.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.232+0000 7fd3eb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3ec0744b0 0x7fd3ec140e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44998/0 (socket says 172.21.15.5:44998) 2024-03-18T15:34:23.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.232+0000 7fd3eb7fe640 1 -- 172.21.15.5:0/2396217480 learned_addr learned my addr 172.21.15.5:0/2396217480 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:23.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.232+0000 7fd3eb7fe640 1 -- 172.21.15.5:0/2396217480 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd3ec0b5450 msgr2=0x7fd3ec13c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:23.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.232+0000 7fd3eb7fe640 1 --2- 172.21.15.5:0/2396217480 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd3ec0b5450 0x7fd3ec13c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:23.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.232+0000 7fd3eb7fe640 1 -- 172.21.15.5:0/2396217480 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd3ec144a50 msgr2=0x7fd3ec143c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:23.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.232+0000 7fd3eb7fe640 1 --2- 172.21.15.5:0/2396217480 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd3ec144a50 0x7fd3ec143c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:23.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.232+0000 7fd3eb7fe640 1 -- 172.21.15.5:0/2396217480 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3ec141d50 con 0x7fd3ec0744b0 2024-03-18T15:34:23.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.232+0000 7fd3eb7fe640 1 --2- 172.21.15.5:0/2396217480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3ec0744b0 0x7fd3ec140e90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc019f90 tx=0x7fd3dc01aba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:23.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.232+0000 7fd3d3fff640 1 -- 172.21.15.5:0/2396217480 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc0b2050 con 0x7fd3ec0744b0 2024-03-18T15:34:23.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.232+0000 7fd3d3fff640 1 -- 172.21.15.5:0/2396217480 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3dc0be070 con 0x7fd3ec0744b0 2024-03-18T15:34:23.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.232+0000 7fd3d3fff640 1 -- 172.21.15.5:0/2396217480 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc0b98b0 con 0x7fd3ec0744b0 2024-03-18T15:34:23.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.232+0000 7fd3f222f640 1 -- 172.21.15.5:0/2396217480 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3ec143730 con 0x7fd3ec0744b0 2024-03-18T15:34:23.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.232+0000 7fd3f222f640 1 -- 172.21.15.5:0/2396217480 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3ec071920 con 0x7fd3ec0744b0 2024-03-18T15:34:23.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.232+0000 7fd3f222f640 1 -- 172.21.15.5:0/2396217480 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3ec04f550 con 0x7fd3ec0744b0 2024-03-18T15:34:23.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.236+0000 7fd3d3fff640 1 -- 172.21.15.5:0/2396217480 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fd3dc0ab070 con 0x7fd3ec0744b0 2024-03-18T15:34:23.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.236+0000 7fd3d3fff640 1 --2- 172.21.15.5:0/2396217480 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd3d8067690 0x7fd3d8069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:23.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.236+0000 7fd3d3fff640 1 -- 172.21.15.5:0/2396217480 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7fd3dc120080 con 0x7fd3ec0744b0 2024-03-18T15:34:23.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.236+0000 7fd3eaffd640 1 --2- 172.21.15.5:0/2396217480 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd3d8067690 0x7fd3d8069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:23.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.236+0000 7fd3eaffd640 1 --2- 172.21.15.5:0/2396217480 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd3d8067690 0x7fd3d8069b50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec071f20 tx=0x7fd3e0007a50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:23.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.244+0000 7fd3d3fff640 1 -- 172.21.15.5:0/2396217480 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fd3ec073530 con 0x7fd3ec0744b0 2024-03-18T15:34:23.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.404+0000 7fd3f222f640 1 -- 172.21.15.5:0/2396217480 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd3ec0b3070 con 0x7fd3ec0744b0 2024-03-18T15:34:23.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.404+0000 7fd3d3fff640 1 -- 172.21.15.5:0/2396217480 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+16526 (secure 0 0 0) 0x7fd3dc016070 con 0x7fd3ec0744b0 2024-03-18T15:34:23.408 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:34:23.408 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":29,"fsid":"ef54fe03-5aa7-4eb2-9b1e-66c9cc345b94","created":"2024-03-18T15:33:18.086614+0000","modified":"2024-03-18T15:34:22.107767+0000","last_up_change":"2024-03-18T15:33:29.229911+0000","last_in_change":"2024-03-18T15:33:20.323840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:28.963186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-18T15:34:20.652674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-18T15:34:21.110594+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df1c754b-af59-4d16-8919-fbb6386b3c48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6804","nonce":976264754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6808","nonce":976264754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6815","nonce":976264754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6811","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6813","nonce":976264754}]},"public_addr":"172.21.15.5:6804/976264754","cluster_addr":"172.21.15.5:6808/976264754","heartbeat_back_addr":"172.21.15.5:6815/976264754","heartbeat_front_addr":"172.21.15.5:6813/976264754","state":["exists","up"]},{"osd":1,"uuid":"b9a08b5e-8af4-4144-bdba-d69ed1ac627a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6801","nonce":3901761126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6803","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6805","nonce":3901761126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6812","nonce":3901761126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6807","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6809","nonce":3901761126}]},"public_addr":"172.21.15.5:6801/3901761126","cluster_addr":"172.21.15.5:6805/3901761126","heartbeat_back_addr":"172.21.15.5:6812/3901761126","heartbeat_front_addr":"172.21.15.5:6809/3901761126","state":["exists","up"]},{"osd":2,"uuid":"c3e8bb76-e382-439d-b380-62ef92f1255a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6817","nonce":502968448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6819","nonce":502968448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6823","nonce":502968448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6821","nonce":502968448}]},"public_addr":"172.21.15.5:6817/502968448","cluster_addr":"172.21.15.5:6819/502968448","heartbeat_back_addr":"172.21.15.5:6823/502968448","heartbeat_front_addr":"172.21.15.5:6821/502968448","state":["exists","up"]},{"osd":3,"uuid":"e8c94625-f2cd-4eba-b53d-2e7356e1c02a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6825","nonce":2425932735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6827","nonce":2425932735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6831","nonce":2425932735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6829","nonce":2425932735}]},"public_addr":"172.21.15.5:6825/2425932735","cluster_addr":"172.21.15.5:6827/2425932735","heartbeat_back_addr":"172.21.15.5:6831/2425932735","heartbeat_front_addr":"172.21.15.5:6829/2425932735","state":["exists","up"]},{"osd":4,"uuid":"0099dd52-6265-4d32-82e4-3ef16e73add0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6817","nonce":976514285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6819","nonce":976514285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6823","nonce":976514285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6821","nonce":976514285}]},"public_addr":"172.21.15.80:6817/976514285","cluster_addr":"172.21.15.80:6819/976514285","heartbeat_back_addr":"172.21.15.80:6823/976514285","heartbeat_front_addr":"172.21.15.80:6821/976514285","state":["exists","up"]},{"osd":5,"uuid":"ed10c84f-8c39-41cc-9b64-7c482a94a48e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6825","nonce":627153447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6827","nonce":627153447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6831","nonce":627153447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6829","nonce":627153447}]},"public_addr":"172.21.15.80:6825/627153447","cluster_addr":"172.21.15.80:6827/627153447","heartbeat_back_addr":"172.21.15.80:6831/627153447","heartbeat_front_addr":"172.21.15.80:6829/627153447","state":["exists","up"]},{"osd":6,"uuid":"c7a3538a-7048-4fba-8781-b440f03c25db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6803","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6805","nonce":488342527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6807","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6809","nonce":488342527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6815","nonce":488342527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6811","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6813","nonce":488342527}]},"public_addr":"172.21.15.80:6805/488342527","cluster_addr":"172.21.15.80:6809/488342527","heartbeat_back_addr":"172.21.15.80:6815/488342527","heartbeat_front_addr":"172.21.15.80:6813/488342527","state":["exists","up"]},{"osd":7,"uuid":"f9418104-3a97-4110-bc3a-1f8151d0f757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6801","nonce":982831666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6804","nonce":982831666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6812","nonce":982831666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6808","nonce":982831666}]},"public_addr":"172.21.15.80:6801/982831666","cluster_addr":"172.21.15.80:6804/982831666","heartbeat_back_addr":"172.21.15.80:6812/982831666","heartbeat_front_addr":"172.21.15.80:6808/982831666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.970828+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.858116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.987556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.864603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.756293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.591502+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:34:23.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.408+0000 7fd3f222f640 1 -- 172.21.15.5:0/2396217480 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd3d8067690 msgr2=0x7fd3d8069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:23.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.408+0000 7fd3f222f640 1 --2- 172.21.15.5:0/2396217480 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd3d8067690 0x7fd3d8069b50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec071f20 tx=0x7fd3e0007a50 comp rx=0 tx=0).stop 2024-03-18T15:34:23.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.408+0000 7fd3f222f640 1 -- 172.21.15.5:0/2396217480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3ec0744b0 msgr2=0x7fd3ec140e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:23.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.408+0000 7fd3f222f640 1 --2- 172.21.15.5:0/2396217480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3ec0744b0 0x7fd3ec140e90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc019f90 tx=0x7fd3dc01aba0 comp rx=0 tx=0).stop 2024-03-18T15:34:23.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.408+0000 7fd3f222f640 1 -- 172.21.15.5:0/2396217480 shutdown_connections 2024-03-18T15:34:23.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.408+0000 7fd3f222f640 1 --2- 172.21.15.5:0/2396217480 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd3d8067690 0x7fd3d8069b50 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:23.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.408+0000 7fd3f222f640 1 --2- 172.21.15.5:0/2396217480 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd3ec144a50 0x7fd3ec143c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:23.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.408+0000 7fd3f222f640 1 --2- 172.21.15.5:0/2396217480 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd3ec0b5450 0x7fd3ec13c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:23.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.408+0000 7fd3f222f640 1 --2- 172.21.15.5:0/2396217480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3ec0744b0 0x7fd3ec140e90 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:23.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.408+0000 7fd3f222f640 1 -- 172.21.15.5:0/2396217480 >> 172.21.15.5:0/2396217480 conn(0x7fd3ec0b9190 msgr2=0x7fd3ec0b40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:23.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.408+0000 7fd3f222f640 1 -- 172.21.15.5:0/2396217480 shutdown_connections 2024-03-18T15:34:23.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.408+0000 7fd3f222f640 1 -- 172.21.15.5:0/2396217480 wait complete. 2024-03-18T15:34:23.426 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-18T15:34:23.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.544+0000 7f6b1f8a8640 1 Processor -- start 2024-03-18T15:34:23.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.544+0000 7f6b1f8a8640 1 -- start start 2024-03-18T15:34:23.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.544+0000 7f6b1f8a8640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6b1806d070 0x7f6b181489a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:23.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.544+0000 7f6b1f8a8640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f6b18148ee0 0x7f6b1814faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:23.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.544+0000 7f6b1f8a8640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b18149370 con 0x7f6b180b3b60 2024-03-18T15:34:23.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.544+0000 7f6b1f8a8640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f6b1806e830 con 0x7f6b1806d070 2024-03-18T15:34:23.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.544+0000 7f6b1f8a8640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f6b1806eb10 con 0x7f6b18148ee0 2024-03-18T15:34:23.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.544+0000 7f6b1d644640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f6b180b3b60 0x7f6b180724c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47026/0 (socket says 172.21.15.5:47026) 2024-03-18T15:34:23.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.544+0000 7f6b1d644640 1 -- 172.21.15.5:0/180851017 learned_addr learned my addr 172.21.15.5:0/180851017 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:23.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.544+0000 7f6b1ce43640 1 --2- 172.21.15.5:0/180851017 >> v2:172.21.15.5:3300/0 conn(0x7f6b1806d070 0x7f6b181489a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:23.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.544+0000 7f6b0ffff640 1 -- 172.21.15.5:0/180851017 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1171188709 0 0) 0x7f6b18149370 con 0x7f6b180b3b60 2024-03-18T15:34:23.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.544+0000 7f6b1de45640 1 --2- 172.21.15.5:0/180851017 >> v2:172.21.15.80:3300/0 conn(0x7f6b18148ee0 0x7f6b1814faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:23.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.544+0000 7f6b0ffff640 1 -- 172.21.15.5:0/180851017 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6b04003530 con 0x7f6b180b3b60 2024-03-18T15:34:23.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.544+0000 7f6b1ce43640 1 -- 172.21.15.5:0/180851017 >> v2:172.21.15.80:3300/0 conn(0x7f6b18148ee0 msgr2=0x7f6b1814faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:23.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.544+0000 7f6b1ce43640 1 --2- 172.21.15.5:0/180851017 >> v2:172.21.15.80:3300/0 conn(0x7f6b18148ee0 0x7f6b1814faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:23.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.544+0000 7f6b1ce43640 1 -- 172.21.15.5:0/180851017 >> v1:172.21.15.5:6789/0 conn(0x7f6b180b3b60 legacy=0x7f6b180724c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:23.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.544+0000 7f6b1ce43640 1 -- 172.21.15.5:0/180851017 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b1806dd10 con 0x7f6b1806d070 2024-03-18T15:34:23.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.544+0000 7f6b1de45640 1 --2- 172.21.15.5:0/180851017 >> v2:172.21.15.80:3300/0 conn(0x7f6b18148ee0 0x7f6b1814faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:23.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.548+0000 7f6b1ce43640 1 --2- 172.21.15.5:0/180851017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b1806d070 0x7f6b181489a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6b180b96f0 tx=0x7f6b0801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9456f8ca6275edbd server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:23.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.548+0000 7f6b0ffff640 1 -- 172.21.15.5:0/180851017 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b080b0070 con 0x7f6b1806d070 2024-03-18T15:34:23.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.548+0000 7f6b0ffff640 1 -- 172.21.15.5:0/180851017 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b080ab600 con 0x7f6b1806d070 2024-03-18T15:34:23.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.548+0000 7f6b0ffff640 1 -- 172.21.15.5:0/180851017 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b080ab9d0 con 0x7f6b1806d070 2024-03-18T15:34:23.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.548+0000 7f6b1f8a8640 1 -- 172.21.15.5:0/180851017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b1806d070 msgr2=0x7f6b181489a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:23.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.548+0000 7f6b1f8a8640 1 --2- 172.21.15.5:0/180851017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b1806d070 0x7f6b181489a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6b180b96f0 tx=0x7f6b0801c3f0 comp rx=0 tx=0).stop 2024-03-18T15:34:23.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.548+0000 7f6b1f8a8640 1 -- 172.21.15.5:0/180851017 shutdown_connections 2024-03-18T15:34:23.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.548+0000 7f6b1f8a8640 1 --2- 172.21.15.5:0/180851017 >> v2:172.21.15.80:3300/0 conn(0x7f6b18148ee0 0x7f6b1814faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:23.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.548+0000 7f6b1f8a8640 1 --2- 172.21.15.5:0/180851017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b1806d070 0x7f6b181489a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:23.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.548+0000 7f6b1f8a8640 1 -- 172.21.15.5:0/180851017 >> 172.21.15.5:0/180851017 conn(0x7f6b180b7a80 msgr2=0x7f6b180b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:23.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.548+0000 7f6b1f8a8640 1 -- 172.21.15.5:0/180851017 shutdown_connections 2024-03-18T15:34:23.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.548+0000 7f6b1f8a8640 1 -- 172.21.15.5:0/180851017 wait complete. 2024-03-18T15:34:23.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.548+0000 7f6b1f8a8640 1 Processor -- start 2024-03-18T15:34:23.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.548+0000 7f6b1f8a8640 1 -- start start 2024-03-18T15:34:23.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.552+0000 7f6b1f8a8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b1806d070 0x7f6b18140ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:23.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.552+0000 7f6b1f8a8640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6b180b3b60 0x7f6b1813ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:23.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.552+0000 7f6b1f8a8640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6b18148ee0 0x7f6b1813e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:23.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.552+0000 7f6b1f8a8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6b18152a60 con 0x7f6b1806d070 2024-03-18T15:34:23.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.552+0000 7f6b1f8a8640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1806e460 con 0x7f6b180b3b60 2024-03-18T15:34:23.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.552+0000 7f6b1f8a8640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f6b18152be0 con 0x7f6b18148ee0 2024-03-18T15:34:23.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.552+0000 7f6b1d644640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b1806d070 0x7f6b18140ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:23.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.552+0000 7f6b1d644640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b1806d070 0x7f6b18140ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45018/0 (socket says 172.21.15.5:45018) 2024-03-18T15:34:23.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.552+0000 7f6b1d644640 1 -- 172.21.15.5:0/459516640 learned_addr learned my addr 172.21.15.5:0/459516640 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:23.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.552+0000 7f6b1ce43640 1 --2- 172.21.15.5:0/459516640 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6b180b3b60 0x7f6b1813ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:23.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.552+0000 7f6b1d644640 1 -- 172.21.15.5:0/459516640 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6b18148ee0 msgr2=0x7f6b1813e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:23.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.552+0000 7f6b1d644640 1 --2- 172.21.15.5:0/459516640 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6b18148ee0 0x7f6b1813e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:23.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.552+0000 7f6b1d644640 1 -- 172.21.15.5:0/459516640 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6b180b3b60 msgr2=0x7f6b1813ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:23.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.552+0000 7f6b1d644640 1 --2- 172.21.15.5:0/459516640 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6b180b3b60 0x7f6b1813ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:23.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.552+0000 7f6b1d644640 1 -- 172.21.15.5:0/459516640 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b1813eb10 con 0x7f6b1806d070 2024-03-18T15:34:23.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.552+0000 7f6b1d644640 1 --2- 172.21.15.5:0/459516640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b1806d070 0x7f6b18140ba0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6b1400cb40 tx=0x7f6b1400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:23.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.552+0000 7f6b0dffb640 1 -- 172.21.15.5:0/459516640 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b14017070 con 0x7f6b1806d070 2024-03-18T15:34:23.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.552+0000 7f6b0dffb640 1 -- 172.21.15.5:0/459516640 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b1400aca0 con 0x7f6b1806d070 2024-03-18T15:34:23.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.552+0000 7f6b1f8a8640 1 -- 172.21.15.5:0/459516640 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b1814f470 con 0x7f6b1806d070 2024-03-18T15:34:23.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.552+0000 7f6b0dffb640 1 -- 172.21.15.5:0/459516640 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b14012620 con 0x7f6b1806d070 2024-03-18T15:34:23.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.552+0000 7f6b1f8a8640 1 -- 172.21.15.5:0/459516640 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b18142670 con 0x7f6b1806d070 2024-03-18T15:34:23.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.552+0000 7f6b1f8a8640 1 -- 172.21.15.5:0/459516640 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b1804f550 con 0x7f6b1806d070 2024-03-18T15:34:23.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.552+0000 7f6b0dffb640 1 -- 172.21.15.5:0/459516640 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f6b1401a070 con 0x7f6b1806d070 2024-03-18T15:34:23.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.552+0000 7f6b0dffb640 1 --2- 172.21.15.5:0/459516640 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6afc067660 0x7f6afc069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:23.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.552+0000 7f6b0dffb640 1 -- 172.21.15.5:0/459516640 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5709+0+0 (secure 0 0 0) 0x7f6b14085650 con 0x7f6b1806d070 2024-03-18T15:34:23.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.552+0000 7f6b1ce43640 1 --2- 172.21.15.5:0/459516640 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6afc067660 0x7f6afc069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:23.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.552+0000 7f6b1ce43640 1 --2- 172.21.15.5:0/459516640 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6afc067660 0x7f6afc069b20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6b080ad000 tx=0x7f6b08008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:23.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.556+0000 7f6b0dffb640 1 -- 172.21.15.5:0/459516640 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f6b1813eb10 con 0x7f6b1806d070 2024-03-18T15:34:23.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:23.752+0000 7f6b1f8a8640 1 -- 172.21.15.5:0/459516640 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f6b180b97b0 con 0x7f6b1806d070 2024-03-18T15:34:24.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.736+0000 7f6b0dffb640 1 -- 172.21.15.5:0/459516640 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7f6b14056360 con 0x7f6b1806d070 2024-03-18T15:34:24.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.740+0000 7f6b1f8a8640 1 -- 172.21.15.5:0/459516640 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6afc067660 msgr2=0x7f6afc069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:24.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.740+0000 7f6b1f8a8640 1 --2- 172.21.15.5:0/459516640 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6afc067660 0x7f6afc069b20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6b080ad000 tx=0x7f6b08008040 comp rx=0 tx=0).stop 2024-03-18T15:34:24.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.740+0000 7f6b1f8a8640 1 -- 172.21.15.5:0/459516640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b1806d070 msgr2=0x7f6b18140ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:24.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.740+0000 7f6b1f8a8640 1 --2- 172.21.15.5:0/459516640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b1806d070 0x7f6b18140ba0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6b1400cb40 tx=0x7f6b1400a060 comp rx=0 tx=0).stop 2024-03-18T15:34:24.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.740+0000 7f6b1f8a8640 1 -- 172.21.15.5:0/459516640 shutdown_connections 2024-03-18T15:34:24.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.740+0000 7f6b1f8a8640 1 --2- 172.21.15.5:0/459516640 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6afc067660 0x7f6afc069b20 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:24.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.740+0000 7f6b1f8a8640 1 --2- 172.21.15.5:0/459516640 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6b18148ee0 0x7f6b1813e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:24.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.740+0000 7f6b1f8a8640 1 --2- 172.21.15.5:0/459516640 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6b180b3b60 0x7f6b1813ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:24.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.740+0000 7f6b1f8a8640 1 --2- 172.21.15.5:0/459516640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b1806d070 0x7f6b18140ba0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:24.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.740+0000 7f6b1f8a8640 1 -- 172.21.15.5:0/459516640 >> 172.21.15.5:0/459516640 conn(0x7f6b180b7a80 msgr2=0x7f6b180b28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:24.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.740+0000 7f6b1f8a8640 1 -- 172.21.15.5:0/459516640 shutdown_connections 2024-03-18T15:34:24.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.740+0000 7f6b1f8a8640 1 -- 172.21.15.5:0/459516640 wait complete. 2024-03-18T15:34:24.761 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-18T15:34:24.762 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-18T15:34:24.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.896+0000 7f195e87d640 1 Processor -- start 2024-03-18T15:34:24.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.896+0000 7f195e87d640 1 -- start start 2024-03-18T15:34:24.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.896+0000 7f195e87d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f19580b5710 0x7f195814a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:24.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.896+0000 7f195e87d640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f195814ae60 0x7f1958140ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:24.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.896+0000 7f195e87d640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19580b9d50 con 0x7f1958141430 2024-03-18T15:34:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.896+0000 7f195e87d640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f19580b4320 con 0x7f19580b5710 2024-03-18T15:34:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.896+0000 7f195e87d640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f19580b4570 con 0x7f195814ae60 2024-03-18T15:34:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.896+0000 7f194ffff640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f195814ae60 0x7f1958140ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.896+0000 7f195ce1a640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f1958141430 0x7f19580b7690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.5:43138/0 (socket says 172.21.15.5:43138) 2024-03-18T15:34:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.896+0000 7f1957fff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f19580b5710 0x7f195814a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:24.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.896+0000 7f195ce1a640 1 -- 172.21.15.5:0/1724022557 learned_addr learned my addr 172.21.15.5:0/1724022557 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.896+0000 7f19577fe640 1 -- 172.21.15.5:0/1724022557 <== mon.1 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2278328689 0 0) 0x7f19580b9d50 con 0x7f1958141430 2024-03-18T15:34:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.896+0000 7f19577fe640 1 -- 172.21.15.5:0/1724022557 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1944003530 con 0x7f1958141430 2024-03-18T15:34:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.896+0000 7f1957fff640 1 -- 172.21.15.5:0/1724022557 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f195814ae60 msgr2=0x7f1958140ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.896+0000 7f1957fff640 1 --2- 172.21.15.5:0/1724022557 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f195814ae60 0x7f1958140ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.896+0000 7f1957fff640 1 -- 172.21.15.5:0/1724022557 >> v1:172.21.15.80:6789/0 conn(0x7f1958141430 legacy=0x7f19580b7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.896+0000 7f1957fff640 1 -- 172.21.15.5:0/1724022557 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19580b4c30 con 0x7f19580b5710 2024-03-18T15:34:24.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.896+0000 7f194ffff640 1 --2- 172.21.15.5:0/1724022557 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f195814ae60 0x7f1958140ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:34:24.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.900+0000 7f1957fff640 1 --2- 172.21.15.5:0/1724022557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f19580b5710 0x7f195814a920 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f19480097f0 tx=0x7f194801c380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71b8ca5a6ef88a8e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:24.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.900+0000 7f19577fe640 1 -- 172.21.15.5:0/1724022557 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19480b0070 con 0x7f19580b5710 2024-03-18T15:34:24.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.900+0000 7f19577fe640 1 -- 172.21.15.5:0/1724022557 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19480ab600 con 0x7f19580b5710 2024-03-18T15:34:24.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.900+0000 7f19577fe640 1 -- 172.21.15.5:0/1724022557 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19480ab9b0 con 0x7f19580b5710 2024-03-18T15:34:24.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.900+0000 7f195e87d640 1 -- 172.21.15.5:0/1724022557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f19580b5710 msgr2=0x7f195814a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:24.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.900+0000 7f195e87d640 1 --2- 172.21.15.5:0/1724022557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f19580b5710 0x7f195814a920 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f19480097f0 tx=0x7f194801c380 comp rx=0 tx=0).stop 2024-03-18T15:34:24.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.900+0000 7f195e87d640 1 -- 172.21.15.5:0/1724022557 shutdown_connections 2024-03-18T15:34:24.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.900+0000 7f195e87d640 1 --2- 172.21.15.5:0/1724022557 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f195814ae60 0x7f1958140ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:24.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.900+0000 7f195e87d640 1 --2- 172.21.15.5:0/1724022557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f19580b5710 0x7f195814a920 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:24.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.900+0000 7f195e87d640 1 -- 172.21.15.5:0/1724022557 >> 172.21.15.5:0/1724022557 conn(0x7f19580673c0 msgr2=0x7f19580677d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:24.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.900+0000 7f195e87d640 1 -- 172.21.15.5:0/1724022557 shutdown_connections 2024-03-18T15:34:24.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.900+0000 7f195e87d640 1 -- 172.21.15.5:0/1724022557 wait complete. 2024-03-18T15:34:24.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.904+0000 7f195e87d640 1 Processor -- start 2024-03-18T15:34:24.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.904+0000 7f195e87d640 1 -- start start 2024-03-18T15:34:24.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.904+0000 7f195e87d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f19580b5710 0x7f19580bb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:24.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.904+0000 7f195e87d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1958141430 0x7f19581e0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:24.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.904+0000 7f195e87d640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f195814ae60 0x7f19581e0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:24.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.904+0000 7f195e87d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f19580b5260 con 0x7f1958141430 2024-03-18T15:34:24.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.904+0000 7f195e87d640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f19580b50e0 con 0x7f19580b5710 2024-03-18T15:34:24.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.904+0000 7f195e87d640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f1958141840 con 0x7f195814ae60 2024-03-18T15:34:24.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.904+0000 7f194ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1958141430 0x7f19581e0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:24.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.904+0000 7f194ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1958141430 0x7f19581e0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45022/0 (socket says 172.21.15.5:45022) 2024-03-18T15:34:24.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.904+0000 7f194ffff640 1 -- 172.21.15.5:0/2977859893 learned_addr learned my addr 172.21.15.5:0/2977859893 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:24.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.904+0000 7f1957fff640 1 --2- 172.21.15.5:0/2977859893 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f19580b5710 0x7f19580bb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:24.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.904+0000 7f195ce1a640 1 --2- 172.21.15.5:0/2977859893 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f195814ae60 0x7f19581e0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:24.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.904+0000 7f194ffff640 1 -- 172.21.15.5:0/2977859893 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f195814ae60 msgr2=0x7f19581e0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:24.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.904+0000 7f194ffff640 1 --2- 172.21.15.5:0/2977859893 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f195814ae60 0x7f19581e0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:24.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.904+0000 7f194ffff640 1 -- 172.21.15.5:0/2977859893 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f19580b5710 msgr2=0x7f19580bb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:24.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.904+0000 7f194ffff640 1 --2- 172.21.15.5:0/2977859893 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f19580b5710 0x7f19580bb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:24.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.904+0000 7f194ffff640 1 -- 172.21.15.5:0/2977859893 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19580bd130 con 0x7f1958141430 2024-03-18T15:34:24.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.904+0000 7f1957fff640 1 --2- 172.21.15.5:0/2977859893 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f19580b5710 0x7f19580bb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:34:24.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.904+0000 7f194ffff640 1 --2- 172.21.15.5:0/2977859893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1958141430 0x7f19581e0220 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f194000e980 tx=0x7f194000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:24.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.904+0000 7f19557fa640 1 -- 172.21.15.5:0/2977859893 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1940017070 con 0x7f1958141430 2024-03-18T15:34:24.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.904+0000 7f19557fa640 1 -- 172.21.15.5:0/2977859893 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f194000ca00 con 0x7f1958141430 2024-03-18T15:34:24.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.904+0000 7f195e87d640 1 -- 172.21.15.5:0/2977859893 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f195813bea0 con 0x7f1958141430 2024-03-18T15:34:24.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.904+0000 7f19557fa640 1 -- 172.21.15.5:0/2977859893 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f194000cdd0 con 0x7f1958141430 2024-03-18T15:34:24.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.904+0000 7f195e87d640 1 -- 172.21.15.5:0/2977859893 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19580bcc00 con 0x7f1958141430 2024-03-18T15:34:24.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.908+0000 7f195e87d640 1 -- 172.21.15.5:0/2977859893 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f195804f550 con 0x7f1958141430 2024-03-18T15:34:24.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.908+0000 7f19557fa640 1 -- 172.21.15.5:0/2977859893 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f194001a070 con 0x7f1958141430 2024-03-18T15:34:24.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.908+0000 7f19557fa640 1 --2- 172.21.15.5:0/2977859893 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f1938067660 0x7f1938069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:24.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.908+0000 7f19557fa640 1 -- 172.21.15.5:0/2977859893 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7f1940085a50 con 0x7f1958141430 2024-03-18T15:34:24.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.908+0000 7f1957fff640 1 --2- 172.21.15.5:0/2977859893 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f1938067660 0x7f1938069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:24.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.908+0000 7f1957fff640 1 --2- 172.21.15.5:0/2977859893 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f1938067660 0x7f1938069b20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f19480ad000 tx=0x7f19480a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:24.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:24.912+0000 7f19557fa640 1 -- 172.21.15.5:0/2977859893 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f1940056760 con 0x7f1958141430 2024-03-18T15:34:25.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.104+0000 7f195e87d640 1 -- 172.21.15.5:0/2977859893 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f195814a4c0 con 0x7f1958141430 2024-03-18T15:34:25.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.764+0000 7f19557fa640 1 -- 172.21.15.5:0/2977859893 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7f1940056060 con 0x7f1958141430 2024-03-18T15:34:25.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.764+0000 7f195e87d640 1 -- 172.21.15.5:0/2977859893 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f1938067660 msgr2=0x7f1938069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:25.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.764+0000 7f195e87d640 1 --2- 172.21.15.5:0/2977859893 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f1938067660 0x7f1938069b20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f19480ad000 tx=0x7f19480a6000 comp rx=0 tx=0).stop 2024-03-18T15:34:25.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.764+0000 7f195e87d640 1 -- 172.21.15.5:0/2977859893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1958141430 msgr2=0x7f19581e0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:25.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.764+0000 7f195e87d640 1 --2- 172.21.15.5:0/2977859893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1958141430 0x7f19581e0220 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f194000e980 tx=0x7f194000ee40 comp rx=0 tx=0).stop 2024-03-18T15:34:25.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.764+0000 7f195e87d640 1 -- 172.21.15.5:0/2977859893 shutdown_connections 2024-03-18T15:34:25.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.764+0000 7f195e87d640 1 --2- 172.21.15.5:0/2977859893 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f1938067660 0x7f1938069b20 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:25.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.764+0000 7f195e87d640 1 --2- 172.21.15.5:0/2977859893 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f195814ae60 0x7f19581e0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:25.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.764+0000 7f195e87d640 1 --2- 172.21.15.5:0/2977859893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1958141430 0x7f19581e0220 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:25.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.764+0000 7f195e87d640 1 --2- 172.21.15.5:0/2977859893 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f19580b5710 0x7f19580bb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:25.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.764+0000 7f195e87d640 1 -- 172.21.15.5:0/2977859893 >> 172.21.15.5:0/2977859893 conn(0x7f19580673c0 msgr2=0x7f19581452a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:25.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.764+0000 7f195e87d640 1 -- 172.21.15.5:0/2977859893 shutdown_connections 2024-03-18T15:34:25.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.764+0000 7f195e87d640 1 -- 172.21.15.5:0/2977859893 wait complete. 2024-03-18T15:34:25.791 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-18T15:34:25.791 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:34:25.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.908+0000 7f706185d640 1 Processor -- start 2024-03-18T15:34:25.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.908+0000 7f706185d640 1 -- start start 2024-03-18T15:34:25.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.908+0000 7f706185d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f705c0af790 0x7f705c072b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:25.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.908+0000 7f706185d640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f705c0b26a0 0x7f705c0b2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:25.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.908+0000 7f706185d640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f705c0b3070 con 0x7f705c06d070 2024-03-18T15:34:25.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.908+0000 7f706185d640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f705c0afb70 con 0x7f705c0af790 2024-03-18T15:34:25.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.908+0000 7f706185d640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f705c0b4a20 con 0x7f705c0b26a0 2024-03-18T15:34:25.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.908+0000 7f705affd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f705c0af790 0x7f705c072b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:25.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.908+0000 7f705a7fc640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f705c06d070 0x7f705c147780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47030/0 (socket says 172.21.15.5:47030) 2024-03-18T15:34:25.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.908+0000 7f705a7fc640 1 -- 172.21.15.5:0/3198480810 learned_addr learned my addr 172.21.15.5:0/3198480810 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:25.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.908+0000 7f705b7fe640 1 --2- 172.21.15.5:0/3198480810 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f705c0b26a0 0x7f705c0b2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:25.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.908+0000 7f705affd640 1 -- 172.21.15.5:0/3198480810 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f705c0b26a0 msgr2=0x7f705c0b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:25.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.908+0000 7f705affd640 1 --2- 172.21.15.5:0/3198480810 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f705c0b26a0 0x7f705c0b2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:25.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.908+0000 7f705affd640 1 -- 172.21.15.5:0/3198480810 >> v1:172.21.15.5:6789/0 conn(0x7f705c06d070 legacy=0x7f705c147780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:25.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.908+0000 7f705affd640 1 -- 172.21.15.5:0/3198480810 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f705c0b4d00 con 0x7f705c0af790 2024-03-18T15:34:25.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.908+0000 7f705b7fe640 1 --2- 172.21.15.5:0/3198480810 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f705c0b26a0 0x7f705c0b2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:25.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.908+0000 7f705affd640 1 --2- 172.21.15.5:0/3198480810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f705c0af790 0x7f705c072b40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f704c0097c0 tx=0x7f704c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c05e0af3bacd1ab2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:25.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.908+0000 7f7059ffb640 1 -- 172.21.15.5:0/3198480810 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c0b0070 con 0x7f705c0af790 2024-03-18T15:34:25.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.912+0000 7f7059ffb640 1 -- 172.21.15.5:0/3198480810 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f704c0ab600 con 0x7f705c0af790 2024-03-18T15:34:25.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.912+0000 7f7059ffb640 1 -- 172.21.15.5:0/3198480810 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c0ab9d0 con 0x7f705c0af790 2024-03-18T15:34:25.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.912+0000 7f706185d640 1 -- 172.21.15.5:0/3198480810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f705c0af790 msgr2=0x7f705c072b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:25.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.912+0000 7f706185d640 1 --2- 172.21.15.5:0/3198480810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f705c0af790 0x7f705c072b40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f704c0097c0 tx=0x7f704c01c410 comp rx=0 tx=0).stop 2024-03-18T15:34:25.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.912+0000 7f706185d640 1 -- 172.21.15.5:0/3198480810 shutdown_connections 2024-03-18T15:34:25.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.912+0000 7f706185d640 1 --2- 172.21.15.5:0/3198480810 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f705c0b26a0 0x7f705c0b2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:25.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.912+0000 7f706185d640 1 --2- 172.21.15.5:0/3198480810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f705c0af790 0x7f705c072b40 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:25.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.912+0000 7f706185d640 1 -- 172.21.15.5:0/3198480810 >> 172.21.15.5:0/3198480810 conn(0x7f705c0b5670 msgr2=0x7f705c0b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:25.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.912+0000 7f706185d640 1 -- 172.21.15.5:0/3198480810 shutdown_connections 2024-03-18T15:34:25.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.912+0000 7f706185d640 1 -- 172.21.15.5:0/3198480810 wait complete. 2024-03-18T15:34:25.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.912+0000 7f706185d640 1 Processor -- start 2024-03-18T15:34:25.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.912+0000 7f706185d640 1 -- start start 2024-03-18T15:34:25.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.912+0000 7f706185d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f705c06d070 0x7f705c13a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:25.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.912+0000 7f706185d640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f705c0af790 0x7f705c1e0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:25.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.912+0000 7f706185d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f705c0b26a0 0x7f705c1e1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:25.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.912+0000 7f706185d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f705c1516a0 con 0x7f705c0b26a0 2024-03-18T15:34:25.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.912+0000 7f706185d640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f705c151520 con 0x7f705c06d070 2024-03-18T15:34:25.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.912+0000 7f706185d640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f705c151820 con 0x7f705c0af790 2024-03-18T15:34:25.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.912+0000 7f705b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f705c0b26a0 0x7f705c1e1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:25.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.912+0000 7f705b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f705c0b26a0 0x7f705c1e1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45044/0 (socket says 172.21.15.5:45044) 2024-03-18T15:34:25.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.912+0000 7f705b7fe640 1 -- 172.21.15.5:0/995294698 learned_addr learned my addr 172.21.15.5:0/995294698 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:25.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.912+0000 7f705affd640 1 --2- 172.21.15.5:0/995294698 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f705c06d070 0x7f705c13a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:25.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.912+0000 7f705a7fc640 1 --2- 172.21.15.5:0/995294698 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f705c0af790 0x7f705c1e0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:25.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.912+0000 7f705b7fe640 1 -- 172.21.15.5:0/995294698 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f705c0af790 msgr2=0x7f705c1e0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:25.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.912+0000 7f705b7fe640 1 --2- 172.21.15.5:0/995294698 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f705c0af790 0x7f705c1e0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:25.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.912+0000 7f705b7fe640 1 -- 172.21.15.5:0/995294698 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f705c06d070 msgr2=0x7f705c13a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:25.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.912+0000 7f705b7fe640 1 --2- 172.21.15.5:0/995294698 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f705c06d070 0x7f705c13a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:25.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.912+0000 7f705b7fe640 1 -- 172.21.15.5:0/995294698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f705c071560 con 0x7f705c0b26a0 2024-03-18T15:34:25.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.912+0000 7f705affd640 1 --2- 172.21.15.5:0/995294698 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f705c06d070 0x7f705c13a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:25.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.912+0000 7f705a7fc640 1 --2- 172.21.15.5:0/995294698 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f705c0af790 0x7f705c1e0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:25.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.912+0000 7f705b7fe640 1 --2- 172.21.15.5:0/995294698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f705c0b26a0 0x7f705c1e1380 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f705400cf20 tx=0x7f705400a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:25.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.912+0000 7f7053fff640 1 -- 172.21.15.5:0/995294698 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f705401a070 con 0x7f705c0b26a0 2024-03-18T15:34:25.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.912+0000 7f7053fff640 1 -- 172.21.15.5:0/995294698 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f705400d840 con 0x7f705c0b26a0 2024-03-18T15:34:25.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.912+0000 7f706185d640 1 -- 172.21.15.5:0/995294698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f705c13e080 con 0x7f705c0b26a0 2024-03-18T15:34:25.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.912+0000 7f7053fff640 1 -- 172.21.15.5:0/995294698 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f705400dc30 con 0x7f705c0b26a0 2024-03-18T15:34:25.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.912+0000 7f706185d640 1 -- 172.21.15.5:0/995294698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f705c13d390 con 0x7f705c0b26a0 2024-03-18T15:34:25.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.916+0000 7f706185d640 1 -- 172.21.15.5:0/995294698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f705c04f550 con 0x7f705c0b26a0 2024-03-18T15:34:25.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.916+0000 7f7053fff640 1 -- 172.21.15.5:0/995294698 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f705401d070 con 0x7f705c0b26a0 2024-03-18T15:34:25.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.916+0000 7f7053fff640 1 --2- 172.21.15.5:0/995294698 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f703c067660 0x7f703c069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:25.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.916+0000 7f7053fff640 1 -- 172.21.15.5:0/995294698 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7f70540860d0 con 0x7f705c0b26a0 2024-03-18T15:34:25.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.916+0000 7f705affd640 1 --2- 172.21.15.5:0/995294698 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f703c067660 0x7f703c069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:25.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.916+0000 7f705affd640 1 --2- 172.21.15.5:0/995294698 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f703c067660 0x7f703c069b20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f704c0ad000 tx=0x7f704c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:25.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:25.920+0000 7f7053fff640 1 -- 172.21.15.5:0/995294698 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f7054056de0 con 0x7f705c0b26a0 2024-03-18T15:34:26.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.116+0000 7f706185d640 1 -- 172.21.15.5:0/995294698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f705c0aec50 con 0x7f705c0b26a0 2024-03-18T15:34:26.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.116+0000 7f7053fff640 1 -- 172.21.15.5:0/995294698 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+3528 (secure 0 0 0) 0x7f70540566e0 con 0x7f705c0b26a0 2024-03-18T15:34:26.121 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:34:26.121 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4619,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/2921781388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":2921781388},{"type":"v1","addr":"172.21.15.80:6833","nonce":2921781388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-18T15:34:22.108585+0000","modified":"2024-03-18T15:34:25.110609+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4968},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4968":{"gid":4968,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6835/2235154713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2235154713},{"type":"v1","addr":"172.21.15.5:6835","nonce":2235154713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4968,"qdb_cluster":[4968]},"id":2}]} 2024-03-18T15:34:26.121 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 16 2024-03-18T15:34:26.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.120+0000 7f706185d640 1 -- 172.21.15.5:0/995294698 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f703c067660 msgr2=0x7f703c069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:26.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.120+0000 7f706185d640 1 --2- 172.21.15.5:0/995294698 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f703c067660 0x7f703c069b20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f704c0ad000 tx=0x7f704c008040 comp rx=0 tx=0).stop 2024-03-18T15:34:26.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.120+0000 7f706185d640 1 -- 172.21.15.5:0/995294698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f705c0b26a0 msgr2=0x7f705c1e1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:26.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.120+0000 7f706185d640 1 --2- 172.21.15.5:0/995294698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f705c0b26a0 0x7f705c1e1380 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f705400cf20 tx=0x7f705400a7d0 comp rx=0 tx=0).stop 2024-03-18T15:34:26.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.120+0000 7f706185d640 1 -- 172.21.15.5:0/995294698 shutdown_connections 2024-03-18T15:34:26.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.120+0000 7f706185d640 1 --2- 172.21.15.5:0/995294698 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f703c067660 0x7f703c069b20 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:26.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.120+0000 7f706185d640 1 --2- 172.21.15.5:0/995294698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f705c0b26a0 0x7f705c1e1380 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:26.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.120+0000 7f706185d640 1 --2- 172.21.15.5:0/995294698 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f705c0af790 0x7f705c1e0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:26.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.120+0000 7f706185d640 1 --2- 172.21.15.5:0/995294698 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f705c06d070 0x7f705c13a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:26.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.120+0000 7f706185d640 1 -- 172.21.15.5:0/995294698 >> 172.21.15.5:0/995294698 conn(0x7f705c0b5670 msgr2=0x7f705c0adab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:26.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.120+0000 7f706185d640 1 -- 172.21.15.5:0/995294698 shutdown_connections 2024-03-18T15:34:26.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.120+0000 7f706185d640 1 -- 172.21.15.5:0/995294698 wait complete. 2024-03-18T15:34:26.138 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:34:26.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.244+0000 7f08db828640 1 Processor -- start 2024-03-18T15:34:26.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.244+0000 7f08db828640 1 -- start start 2024-03-18T15:34:26.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.248+0000 7f08db828640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f08d414b660 0x7f08d4070360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:26.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.248+0000 7f08db828640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f08d41410d0 0x7f08d4141560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:26.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.248+0000 7f08db828640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08d406b8c0 con 0x7f08d406b4e0 2024-03-18T15:34:26.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.248+0000 7f08db828640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f08d414b1d0 con 0x7f08d41410d0 2024-03-18T15:34:26.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.248+0000 7f08db828640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f08d4071630 con 0x7f08d414b660 2024-03-18T15:34:26.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.248+0000 7f08d95c4640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f08d406b4e0 0x7f08d4071da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47046/0 (socket says 172.21.15.5:47046) 2024-03-18T15:34:26.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.248+0000 7f08d95c4640 1 -- 172.21.15.5:0/3655821321 learned_addr learned my addr 172.21.15.5:0/3655821321 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:26.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.248+0000 7f08d8dc3640 1 --2- 172.21.15.5:0/3655821321 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f08d414b660 0x7f08d4070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:26.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.248+0000 7f08d9dc5640 1 --2- 172.21.15.5:0/3655821321 >> v2:172.21.15.80:3300/0 conn(0x7f08d41410d0 0x7f08d4141560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:26.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.248+0000 7f08cbfff640 1 -- 172.21.15.5:0/3655821321 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2745925631 0 0) 0x7f08d406b8c0 con 0x7f08d406b4e0 2024-03-18T15:34:26.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.248+0000 7f08cbfff640 1 -- 172.21.15.5:0/3655821321 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f08c0003530 con 0x7f08d406b4e0 2024-03-18T15:34:26.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.248+0000 7f08cbfff640 1 -- 172.21.15.5:0/3655821321 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4271288359 0 0) 0x7f08c0003530 con 0x7f08d406b4e0 2024-03-18T15:34:26.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.248+0000 7f08cbfff640 1 -- 172.21.15.5:0/3655821321 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f08d414b660 msgr2=0x7f08d4070360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:26.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.248+0000 7f08cbfff640 1 --2- 172.21.15.5:0/3655821321 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f08d414b660 0x7f08d4070360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:26.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.248+0000 7f08cbfff640 1 -- 172.21.15.5:0/3655821321 >> v2:172.21.15.80:3300/0 conn(0x7f08d41410d0 msgr2=0x7f08d4141560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:26.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.248+0000 7f08cbfff640 1 --2- 172.21.15.5:0/3655821321 >> v2:172.21.15.80:3300/0 conn(0x7f08d41410d0 0x7f08d4141560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:26.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.248+0000 7f08cbfff640 1 -- 172.21.15.5:0/3655821321 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08d414ab80 con 0x7f08d406b4e0 2024-03-18T15:34:26.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.248+0000 7f08cbfff640 1 -- 172.21.15.5:0/3655821321 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3837549003 0 0) 0x7f08d0001680 con 0x7f08d406b4e0 2024-03-18T15:34:26.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.248+0000 7f08cbfff640 1 -- 172.21.15.5:0/3655821321 >> v1:172.21.15.5:6789/0 conn(0x7f08d406b4e0 legacy=0x7f08d4071da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:26.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.248+0000 7f08cbfff640 1 --2- 172.21.15.5:0/3655821321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08c0003aa0 0x7f08c0003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:26.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.248+0000 7f08cbfff640 1 --2- 172.21.15.5:0/3655821321 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f08c0004810 0x7f08c0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:26.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.248+0000 7f08cbfff640 1 --2- 172.21.15.5:0/3655821321 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f08d414b660 0x7f08c00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:26.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.248+0000 7f08cbfff640 1 -- 172.21.15.5:0/3655821321 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f08d414b1d0 con 0x7f08c0003aa0 2024-03-18T15:34:26.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.248+0000 7f08cbfff640 1 -- 172.21.15.5:0/3655821321 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f08d4071630 con 0x7f08c0004810 2024-03-18T15:34:26.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.248+0000 7f08cbfff640 1 -- 172.21.15.5:0/3655821321 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f08c000bab0 con 0x7f08d414b660 2024-03-18T15:34:26.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.248+0000 7f08d95c4640 1 --2- 172.21.15.5:0/3655821321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08c0003aa0 0x7f08c0003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:26.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.248+0000 7f08d95c4640 1 -- 172.21.15.5:0/3655821321 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f08d414b660 msgr2=0x7f08c00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:26.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.248+0000 7f08d95c4640 1 --2- 172.21.15.5:0/3655821321 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f08d414b660 0x7f08c00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:26.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.248+0000 7f08d95c4640 1 -- 172.21.15.5:0/3655821321 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f08c0004810 msgr2=0x7f08c0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:26.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.248+0000 7f08d95c4640 1 --2- 172.21.15.5:0/3655821321 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f08c0004810 0x7f08c0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:26.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.248+0000 7f08d95c4640 1 -- 172.21.15.5:0/3655821321 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08c0003530 con 0x7f08c0003aa0 2024-03-18T15:34:26.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.252+0000 7f08d95c4640 1 --2- 172.21.15.5:0/3655821321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08c0003aa0 0x7f08c0003ec0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f08d0001060 tx=0x7f08d001cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd17e4c962fe0de6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:26.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.252+0000 7f08cbfff640 1 -- 172.21.15.5:0/3655821321 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d0007d60 con 0x7f08c0003aa0 2024-03-18T15:34:26.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.252+0000 7f08cbfff640 1 -- 172.21.15.5:0/3655821321 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08d00a9020 con 0x7f08c0003aa0 2024-03-18T15:34:26.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.252+0000 7f08cbfff640 1 -- 172.21.15.5:0/3655821321 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d0005b20 con 0x7f08c0003aa0 2024-03-18T15:34:26.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.252+0000 7f08db828640 1 -- 172.21.15.5:0/3655821321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08c0003aa0 msgr2=0x7f08c0003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:26.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.252+0000 7f08db828640 1 --2- 172.21.15.5:0/3655821321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08c0003aa0 0x7f08c0003ec0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f08d0001060 tx=0x7f08d001cbe0 comp rx=0 tx=0).stop 2024-03-18T15:34:26.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.252+0000 7f08d9dc5640 1 -- 172.21.15.5:0/3655821321 reap_dead start 2024-03-18T15:34:26.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.252+0000 7f08db828640 1 -- 172.21.15.5:0/3655821321 shutdown_connections 2024-03-18T15:34:26.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.252+0000 7f08db828640 1 -- 172.21.15.5:0/3655821321 >> 172.21.15.5:0/3655821321 conn(0x7f08d40b6680 msgr2=0x7f08d40b6a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:26.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.252+0000 7f08db828640 1 -- 172.21.15.5:0/3655821321 shutdown_connections 2024-03-18T15:34:26.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.252+0000 7f08db828640 1 -- 172.21.15.5:0/3655821321 wait complete. 2024-03-18T15:34:26.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.252+0000 7f08db828640 1 Processor -- start 2024-03-18T15:34:26.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.252+0000 7f08db828640 1 -- start start 2024-03-18T15:34:26.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.252+0000 7f08db828640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f08d4150610 0x7f08d40b8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:26.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.252+0000 7f08db828640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f08d414a420 0x7f08d4148890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:26.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.252+0000 7f08db828640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08d4138c20 0x7f08d41390a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:26.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.252+0000 7f08db828640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f08d4141c20 con 0x7f08d4138c20 2024-03-18T15:34:26.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.252+0000 7f08db828640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f08d4141aa0 con 0x7f08d4150610 2024-03-18T15:34:26.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.252+0000 7f08db828640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f08d4152580 con 0x7f08d414a420 2024-03-18T15:34:26.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.252+0000 7f08d9dc5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08d4138c20 0x7f08d41390a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:26.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.252+0000 7f08d9dc5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08d4138c20 0x7f08d41390a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45056/0 (socket says 172.21.15.5:45056) 2024-03-18T15:34:26.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.252+0000 7f08d9dc5640 1 -- 172.21.15.5:0/3286955986 learned_addr learned my addr 172.21.15.5:0/3286955986 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:26.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.252+0000 7f08d95c4640 1 --2- 172.21.15.5:0/3286955986 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f08d4150610 0x7f08d40b8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:26.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.252+0000 7f08d8dc3640 1 --2- 172.21.15.5:0/3286955986 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f08d414a420 0x7f08d4148890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:26.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.252+0000 7f08d9dc5640 1 -- 172.21.15.5:0/3286955986 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f08d414a420 msgr2=0x7f08d4148890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:26.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.252+0000 7f08d9dc5640 1 --2- 172.21.15.5:0/3286955986 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f08d414a420 0x7f08d4148890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:26.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.252+0000 7f08d9dc5640 1 -- 172.21.15.5:0/3286955986 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f08d4150610 msgr2=0x7f08d40b8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:26.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.252+0000 7f08d9dc5640 1 --2- 172.21.15.5:0/3286955986 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f08d4150610 0x7f08d40b8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:26.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.252+0000 7f08d9dc5640 1 -- 172.21.15.5:0/3286955986 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08d40b7c10 con 0x7f08d4138c20 2024-03-18T15:34:26.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.252+0000 7f08d8dc3640 1 --2- 172.21.15.5:0/3286955986 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f08d414a420 0x7f08d4148890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:26.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.252+0000 7f08d9dc5640 1 --2- 172.21.15.5:0/3286955986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08d4138c20 0x7f08d41390a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f08d40b4240 tx=0x7f08cc00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:26.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.252+0000 7f08c9ffb640 1 -- 172.21.15.5:0/3286955986 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc018020 con 0x7f08d4138c20 2024-03-18T15:34:26.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.252+0000 7f08c9ffb640 1 -- 172.21.15.5:0/3286955986 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08cc00bb10 con 0x7f08d4138c20 2024-03-18T15:34:26.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.252+0000 7f08c9ffb640 1 -- 172.21.15.5:0/3286955986 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc014400 con 0x7f08d4138c20 2024-03-18T15:34:26.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.252+0000 7f08db828640 1 -- 172.21.15.5:0/3286955986 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08d41509f0 con 0x7f08d4138c20 2024-03-18T15:34:26.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.252+0000 7f08db828640 1 -- 172.21.15.5:0/3286955986 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08d40b9600 con 0x7f08d4138c20 2024-03-18T15:34:26.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.256+0000 7f08db828640 1 -- 172.21.15.5:0/3286955986 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08d414a800 con 0x7f08d4138c20 2024-03-18T15:34:26.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.256+0000 7f08c9ffb640 1 -- 172.21.15.5:0/3286955986 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f08cc01b020 con 0x7f08d4138c20 2024-03-18T15:34:26.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.256+0000 7f08c9ffb640 1 --2- 172.21.15.5:0/3286955986 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f08b8067690 0x7f08b8069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:26.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.256+0000 7f08c9ffb640 1 -- 172.21.15.5:0/3286955986 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7f08cc085ca0 con 0x7f08d4138c20 2024-03-18T15:34:26.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.256+0000 7f08d95c4640 1 --2- 172.21.15.5:0/3286955986 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f08b8067690 0x7f08b8069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:26.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.256+0000 7f08d95c4640 1 --2- 172.21.15.5:0/3286955986 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f08b8067690 0x7f08b8069b50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f08d00aa000 tx=0x7f08d0000ff0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:26.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.264+0000 7f08c9ffb640 1 -- 172.21.15.5:0/3286955986 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f08d414a800 con 0x7f08d4138c20 2024-03-18T15:34:26.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.420+0000 7f08db828640 1 -- 172.21.15.5:0/3286955986 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f08d404f550 con 0x7f08d4138c20 2024-03-18T15:34:26.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.424+0000 7f08c9ffb640 1 -- 172.21.15.5:0/3286955986 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) ==== 74+0+16513 (secure 0 0 0) 0x7f08cc022030 con 0x7f08d4138c20 2024-03-18T15:34:26.425 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:34:26.426 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":30,"fsid":"ef54fe03-5aa7-4eb2-9b1e-66c9cc345b94","created":"2024-03-18T15:33:18.086614+0000","modified":"2024-03-18T15:34:23.736672+0000","last_up_change":"2024-03-18T15:33:29.229911+0000","last_in_change":"2024-03-18T15:33:20.323840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:28.963186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-18T15:34:20.652674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-18T15:34:21.110594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df1c754b-af59-4d16-8919-fbb6386b3c48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6804","nonce":976264754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6808","nonce":976264754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6815","nonce":976264754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6811","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6813","nonce":976264754}]},"public_addr":"172.21.15.5:6804/976264754","cluster_addr":"172.21.15.5:6808/976264754","heartbeat_back_addr":"172.21.15.5:6815/976264754","heartbeat_front_addr":"172.21.15.5:6813/976264754","state":["exists","up"]},{"osd":1,"uuid":"b9a08b5e-8af4-4144-bdba-d69ed1ac627a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6801","nonce":3901761126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6803","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6805","nonce":3901761126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6812","nonce":3901761126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6807","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6809","nonce":3901761126}]},"public_addr":"172.21.15.5:6801/3901761126","cluster_addr":"172.21.15.5:6805/3901761126","heartbeat_back_addr":"172.21.15.5:6812/3901761126","heartbeat_front_addr":"172.21.15.5:6809/3901761126","state":["exists","up"]},{"osd":2,"uuid":"c3e8bb76-e382-439d-b380-62ef92f1255a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6817","nonce":502968448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6819","nonce":502968448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6823","nonce":502968448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6821","nonce":502968448}]},"public_addr":"172.21.15.5:6817/502968448","cluster_addr":"172.21.15.5:6819/502968448","heartbeat_back_addr":"172.21.15.5:6823/502968448","heartbeat_front_addr":"172.21.15.5:6821/502968448","state":["exists","up"]},{"osd":3,"uuid":"e8c94625-f2cd-4eba-b53d-2e7356e1c02a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6825","nonce":2425932735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6827","nonce":2425932735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6831","nonce":2425932735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6829","nonce":2425932735}]},"public_addr":"172.21.15.5:6825/2425932735","cluster_addr":"172.21.15.5:6827/2425932735","heartbeat_back_addr":"172.21.15.5:6831/2425932735","heartbeat_front_addr":"172.21.15.5:6829/2425932735","state":["exists","up"]},{"osd":4,"uuid":"0099dd52-6265-4d32-82e4-3ef16e73add0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6817","nonce":976514285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6819","nonce":976514285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6823","nonce":976514285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6821","nonce":976514285}]},"public_addr":"172.21.15.80:6817/976514285","cluster_addr":"172.21.15.80:6819/976514285","heartbeat_back_addr":"172.21.15.80:6823/976514285","heartbeat_front_addr":"172.21.15.80:6821/976514285","state":["exists","up"]},{"osd":5,"uuid":"ed10c84f-8c39-41cc-9b64-7c482a94a48e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6825","nonce":627153447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6827","nonce":627153447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6831","nonce":627153447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6829","nonce":627153447}]},"public_addr":"172.21.15.80:6825/627153447","cluster_addr":"172.21.15.80:6827/627153447","heartbeat_back_addr":"172.21.15.80:6831/627153447","heartbeat_front_addr":"172.21.15.80:6829/627153447","state":["exists","up"]},{"osd":6,"uuid":"c7a3538a-7048-4fba-8781-b440f03c25db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6803","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6805","nonce":488342527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6807","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6809","nonce":488342527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6815","nonce":488342527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6811","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6813","nonce":488342527}]},"public_addr":"172.21.15.80:6805/488342527","cluster_addr":"172.21.15.80:6809/488342527","heartbeat_back_addr":"172.21.15.80:6815/488342527","heartbeat_front_addr":"172.21.15.80:6813/488342527","state":["exists","up"]},{"osd":7,"uuid":"f9418104-3a97-4110-bc3a-1f8151d0f757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6801","nonce":982831666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6804","nonce":982831666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6812","nonce":982831666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6808","nonce":982831666}]},"public_addr":"172.21.15.80:6801/982831666","cluster_addr":"172.21.15.80:6804/982831666","heartbeat_back_addr":"172.21.15.80:6812/982831666","heartbeat_front_addr":"172.21.15.80:6808/982831666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.970828+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.858116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.987556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.864603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.756293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.591502+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:34:26.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.424+0000 7f08db828640 1 -- 172.21.15.5:0/3286955986 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f08b8067690 msgr2=0x7f08b8069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:26.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.424+0000 7f08db828640 1 --2- 172.21.15.5:0/3286955986 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f08b8067690 0x7f08b8069b50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f08d00aa000 tx=0x7f08d0000ff0 comp rx=0 tx=0).stop 2024-03-18T15:34:26.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.424+0000 7f08db828640 1 -- 172.21.15.5:0/3286955986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08d4138c20 msgr2=0x7f08d41390a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:26.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.424+0000 7f08db828640 1 --2- 172.21.15.5:0/3286955986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08d4138c20 0x7f08d41390a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f08d40b4240 tx=0x7f08cc00b480 comp rx=0 tx=0).stop 2024-03-18T15:34:26.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.424+0000 7f08db828640 1 -- 172.21.15.5:0/3286955986 shutdown_connections 2024-03-18T15:34:26.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.424+0000 7f08db828640 1 --2- 172.21.15.5:0/3286955986 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f08b8067690 0x7f08b8069b50 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:26.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.424+0000 7f08db828640 1 --2- 172.21.15.5:0/3286955986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08d4138c20 0x7f08d41390a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:26.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.424+0000 7f08db828640 1 --2- 172.21.15.5:0/3286955986 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f08d414a420 0x7f08d4148890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:26.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.424+0000 7f08db828640 1 --2- 172.21.15.5:0/3286955986 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f08d4150610 0x7f08d40b8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:26.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.424+0000 7f08db828640 1 -- 172.21.15.5:0/3286955986 >> 172.21.15.5:0/3286955986 conn(0x7f08d40b6680 msgr2=0x7f08d40690d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:26.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.424+0000 7f08db828640 1 -- 172.21.15.5:0/3286955986 shutdown_connections 2024-03-18T15:34:26.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.424+0000 7f08db828640 1 -- 172.21.15.5:0/3286955986 wait complete. 2024-03-18T15:34:26.443 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-18T15:34:26.443 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-18T15:34:26.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.548+0000 7f4342242640 1 Processor -- start 2024-03-18T15:34:26.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.548+0000 7f4342242640 1 -- start start 2024-03-18T15:34:26.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.548+0000 7f4342242640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f433c074650 0x7f433c1426f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:26.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.548+0000 7f4342242640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f433c06fba0 con 0x7f433c074270 2024-03-18T15:34:26.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.548+0000 7f4342242640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f433c070d10 con 0x7f433c0707c0 2024-03-18T15:34:26.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.548+0000 7f4342242640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f433c070ff0 con 0x7f433c074650 2024-03-18T15:34:26.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.548+0000 7f433b7fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f433c074270 0x7f433c145390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47058/0 (socket says 172.21.15.5:47058) 2024-03-18T15:34:26.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.548+0000 7f433b7fe640 1 -- 172.21.15.5:0/3723780421 learned_addr learned my addr 172.21.15.5:0/3723780421 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:26.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.548+0000 7f433bfff640 1 --2- 172.21.15.5:0/3723780421 >> v2:172.21.15.5:3300/0 conn(0x7f433c074650 0x7f433c1426f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:26.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.548+0000 7f433bfff640 1 -- 172.21.15.5:0/3723780421 >> v1:172.21.15.80:6789/0 conn(0x7f433c0707c0 legacy=0x7f433c141fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:26.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.548+0000 7f433bfff640 1 -- 172.21.15.5:0/3723780421 >> v1:172.21.15.5:6789/0 conn(0x7f433c074270 legacy=0x7f433c145390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:26.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.548+0000 7f433a7fc640 1 -- 172.21.15.5:0/3723780421 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3264658224 0 0) 0x7f433c06fba0 con 0x7f433c074270 2024-03-18T15:34:26.552 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.548+0000 7f433bfff640 1 -- 172.21.15.5:0/3723780421 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f433c0712d0 con 0x7f433c074650 2024-03-18T15:34:26.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.552+0000 7f433bfff640 1 --2- 172.21.15.5:0/3723780421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f433c074650 0x7f433c1426f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4334009d90 tx=0x7f433401c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d996c3cf0bcd03c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:26.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.552+0000 7f433a7fc640 1 -- 172.21.15.5:0/3723780421 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43340b2070 con 0x7f433c074650 2024-03-18T15:34:26.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.552+0000 7f433a7fc640 1 -- 172.21.15.5:0/3723780421 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43340a58e0 con 0x7f433c074650 2024-03-18T15:34:26.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.552+0000 7f433a7fc640 1 -- 172.21.15.5:0/3723780421 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43340a5cb0 con 0x7f433c074650 2024-03-18T15:34:26.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.552+0000 7f4342242640 1 -- 172.21.15.5:0/3723780421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f433c074650 msgr2=0x7f433c1426f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:26.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.552+0000 7f4342242640 1 --2- 172.21.15.5:0/3723780421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f433c074650 0x7f433c1426f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4334009d90 tx=0x7f433401c820 comp rx=0 tx=0).stop 2024-03-18T15:34:26.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.552+0000 7f4342242640 1 -- 172.21.15.5:0/3723780421 shutdown_connections 2024-03-18T15:34:26.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.552+0000 7f4342242640 1 --2- 172.21.15.5:0/3723780421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f433c074650 0x7f433c1426f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:26.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.552+0000 7f4342242640 1 -- 172.21.15.5:0/3723780421 >> 172.21.15.5:0/3723780421 conn(0x7f433c06bad0 msgr2=0x7f433c0b9190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:26.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.552+0000 7f4342242640 1 -- 172.21.15.5:0/3723780421 shutdown_connections 2024-03-18T15:34:26.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.552+0000 7f4342242640 1 -- 172.21.15.5:0/3723780421 wait complete. 2024-03-18T15:34:26.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.552+0000 7f4342242640 1 Processor -- start 2024-03-18T15:34:26.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.552+0000 7f4342242640 1 -- start start 2024-03-18T15:34:26.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.552+0000 7f4342242640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f433c0707c0 0x7f433c0aa410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:26.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.552+0000 7f4342242640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f433c074270 0x7f433c0a9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:26.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.552+0000 7f4342242640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f433c074650 0x7f433c0aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:26.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.552+0000 7f4342242640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f433c1537f0 con 0x7f433c0707c0 2024-03-18T15:34:26.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.552+0000 7f4342242640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f433c153670 con 0x7f433c074270 2024-03-18T15:34:26.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.552+0000 7f4342242640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f433c153970 con 0x7f433c074650 2024-03-18T15:34:26.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.552+0000 7f433b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f433c0707c0 0x7f433c0aa410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:26.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.552+0000 7f433b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f433c0707c0 0x7f433c0aa410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45076/0 (socket says 172.21.15.5:45076) 2024-03-18T15:34:26.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.552+0000 7f433b7fe640 1 -- 172.21.15.5:0/2365278546 learned_addr learned my addr 172.21.15.5:0/2365278546 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:26.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.552+0000 7f433bfff640 1 --2- 172.21.15.5:0/2365278546 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f433c074650 0x7f433c0aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:26.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.552+0000 7f433b7fe640 1 -- 172.21.15.5:0/2365278546 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f433c074650 msgr2=0x7f433c0aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:26.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.552+0000 7f433b7fe640 1 --2- 172.21.15.5:0/2365278546 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f433c074650 0x7f433c0aaca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:26.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.552+0000 7f433b7fe640 1 -- 172.21.15.5:0/2365278546 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f433c074270 msgr2=0x7f433c0a9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:26.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.552+0000 7f433b7fe640 1 --2- 172.21.15.5:0/2365278546 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f433c074270 0x7f433c0a9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:26.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.552+0000 7f433b7fe640 1 -- 172.21.15.5:0/2365278546 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f433c141040 con 0x7f433c0707c0 2024-03-18T15:34:26.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.552+0000 7f433b7fe640 1 --2- 172.21.15.5:0/2365278546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f433c0707c0 0x7f433c0aa410 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f432c00c4d0 tx=0x7f432c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:26.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.552+0000 7f4323fff640 1 -- 172.21.15.5:0/2365278546 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432c018070 con 0x7f433c0707c0 2024-03-18T15:34:26.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.552+0000 7f4323fff640 1 -- 172.21.15.5:0/2365278546 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f432c0075a0 con 0x7f433c0707c0 2024-03-18T15:34:26.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.552+0000 7f4342242640 1 -- 172.21.15.5:0/2365278546 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f433c142100 con 0x7f433c0707c0 2024-03-18T15:34:26.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.552+0000 7f4323fff640 1 -- 172.21.15.5:0/2365278546 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432c004a30 con 0x7f433c0707c0 2024-03-18T15:34:26.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.552+0000 7f4342242640 1 -- 172.21.15.5:0/2365278546 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f433c0aa010 con 0x7f433c0707c0 2024-03-18T15:34:26.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.556+0000 7f4342242640 1 -- 172.21.15.5:0/2365278546 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f433c04f550 con 0x7f433c0707c0 2024-03-18T15:34:26.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.556+0000 7f4323fff640 1 -- 172.21.15.5:0/2365278546 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f432c01b070 con 0x7f433c0707c0 2024-03-18T15:34:26.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.556+0000 7f4323fff640 1 --2- 172.21.15.5:0/2365278546 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f4328067660 0x7f4328069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:26.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.556+0000 7f4323fff640 1 -- 172.21.15.5:0/2365278546 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7f432c084af0 con 0x7f433c0707c0 2024-03-18T15:34:26.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.556+0000 7f433affd640 1 --2- 172.21.15.5:0/2365278546 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f4328067660 0x7f4328069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:26.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.556+0000 7f433affd640 1 --2- 172.21.15.5:0/2365278546 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f4328067660 0x7f4328069b20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f433c144480 tx=0x7f43300079b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:26.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.560+0000 7f4323fff640 1 -- 172.21.15.5:0/2365278546 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f432c055800 con 0x7f433c0707c0 2024-03-18T15:34:26.693 INFO:tasks.ceph.mds.a.smithi005.stderr:2024-03-18T15:34:26.688+0000 7fb984838640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-18T15:34:26.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.708+0000 7f4342242640 1 -- 172.21.15.5:0/2365278546 --> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f433c141d00 con 0x7f4328067660 2024-03-18T15:34:26.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.716+0000 7f4323fff640 1 -- 172.21.15.5:0/2365278546 <== mgr.4110 v2:172.21.15.5:6832/3362034860 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248726 (secure 0 0 0) 0x7f4330005080 con 0x7f4328067660 2024-03-18T15:34:26.722 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:34:26.723 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-03-18T15:34:26.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.720+0000 7f4342242640 1 -- 172.21.15.5:0/2365278546 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f4328067660 msgr2=0x7f4328069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:26.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.720+0000 7f4342242640 1 --2- 172.21.15.5:0/2365278546 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f4328067660 0x7f4328069b20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f433c144480 tx=0x7f43300079b0 comp rx=0 tx=0).stop 2024-03-18T15:34:26.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.720+0000 7f4342242640 1 -- 172.21.15.5:0/2365278546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f433c0707c0 msgr2=0x7f433c0aa410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:26.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.720+0000 7f4342242640 1 --2- 172.21.15.5:0/2365278546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f433c0707c0 0x7f433c0aa410 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f432c00c4d0 tx=0x7f432c00c990 comp rx=0 tx=0).stop 2024-03-18T15:34:26.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.720+0000 7f4342242640 1 -- 172.21.15.5:0/2365278546 shutdown_connections 2024-03-18T15:34:26.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.720+0000 7f4342242640 1 --2- 172.21.15.5:0/2365278546 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f4328067660 0x7f4328069b20 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:26.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.720+0000 7f4342242640 1 --2- 172.21.15.5:0/2365278546 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f433c074650 0x7f433c0aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:26.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.720+0000 7f4342242640 1 --2- 172.21.15.5:0/2365278546 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f433c074270 0x7f433c0a9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:26.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.720+0000 7f4342242640 1 --2- 172.21.15.5:0/2365278546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f433c0707c0 0x7f433c0aa410 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:26.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.720+0000 7f4342242640 1 -- 172.21.15.5:0/2365278546 >> 172.21.15.5:0/2365278546 conn(0x7f433c06bad0 msgr2=0x7f433c0b4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:26.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.720+0000 7f4342242640 1 -- 172.21.15.5:0/2365278546 shutdown_connections 2024-03-18T15:34:26.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.720+0000 7f4342242640 1 -- 172.21.15.5:0/2365278546 wait complete. 2024-03-18T15:34:26.738 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":51,"stamp":"2024-03-18T15:34:24.897408+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592106,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":196,"num_write_kb":2428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149,"ondisk_log_size":149,"up":154,"acting":154,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":80,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":253104,"kb_used_data":3624,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754721616,"statfs":{"total":773094113280,"available":772834934784,"internally_reserved":0,"allocated":3710976,"data_stored":2121014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1738,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.172606"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-18T15:34:21.723958+0000","last_change":"2024-03-18T15:34:21.723958+0000","last_active":"2024-03-18T15:34:21.723958+0000","last_peered":"2024-03-18T15:34:21.723958+0000","last_clean":"2024-03-18T15:34:21.723958+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-18T15:34:21.723958+0000","last_undegraded":"2024-03-18T15:34:21.723958+0000","last_fullsized":"2024-03-18T15:34:21.723958+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-18T15:34:22.783118+0000","last_change":"2024-03-18T15:34:22.774732+0000","last_active":"2024-03-18T15:34:22.783118+0000","last_peered":"2024-03-18T15:34:22.783118+0000","last_clean":"2024-03-18T15:34:22.783118+0000","last_became_active":"2024-03-18T15:34:22.771881+0000","last_became_peered":"2024-03-18T15:34:22.771881+0000","last_unstale":"2024-03-18T15:34:22.783118+0000","last_undegraded":"2024-03-18T15:34:22.783118+0000","last_fullsized":"2024-03-18T15:34:22.783118+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:17:02.398404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-18T15:34:21.723958+0000","last_change":"2024-03-18T15:34:21.723958+0000","last_active":"2024-03-18T15:34:21.723958+0000","last_peered":"2024-03-18T15:34:21.723958+0000","last_clean":"2024-03-18T15:34:21.723958+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-18T15:34:21.723958+0000","last_undegraded":"2024-03-18T15:34:21.723958+0000","last_fullsized":"2024-03-18T15:34:21.723958+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.751835+0000","last_change":"2024-03-18T15:34:22.777889+0000","last_active":"2024-03-18T15:34:23.751835+0000","last_peered":"2024-03-18T15:34:23.751835+0000","last_clean":"2024-03-18T15:34:23.751835+0000","last_became_active":"2024-03-18T15:34:22.777438+0000","last_became_peered":"2024-03-18T15:34:22.777438+0000","last_unstale":"2024-03-18T15:34:23.751835+0000","last_undegraded":"2024-03-18T15:34:23.751835+0000","last_fullsized":"2024-03-18T15:34:23.751835+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:16:42.700006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.742750+0000","last_change":"2024-03-18T15:34:22.758037+0000","last_active":"2024-03-18T15:34:23.742750+0000","last_peered":"2024-03-18T15:34:23.742750+0000","last_clean":"2024-03-18T15:34:23.742750+0000","last_became_active":"2024-03-18T15:34:22.757761+0000","last_became_peered":"2024-03-18T15:34:22.757761+0000","last_unstale":"2024-03-18T15:34:23.742750+0000","last_undegraded":"2024-03-18T15:34:23.742750+0000","last_fullsized":"2024-03-18T15:34:23.742750+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:17:47.611508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-18T15:34:22.783560+0000","last_change":"2024-03-18T15:34:22.780952+0000","last_active":"2024-03-18T15:34:22.783560+0000","last_peered":"2024-03-18T15:34:22.783560+0000","last_clean":"2024-03-18T15:34:22.783560+0000","last_became_active":"2024-03-18T15:34:22.780667+0000","last_became_peered":"2024-03-18T15:34:22.780667+0000","last_unstale":"2024-03-18T15:34:22.783560+0000","last_undegraded":"2024-03-18T15:34:22.783560+0000","last_fullsized":"2024-03-18T15:34:22.783560+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:25:38.946261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.744057+0000","last_change":"2024-03-18T15:34:22.786411+0000","last_active":"2024-03-18T15:34:23.744057+0000","last_peered":"2024-03-18T15:34:23.744057+0000","last_clean":"2024-03-18T15:34:23.744057+0000","last_became_active":"2024-03-18T15:34:22.785695+0000","last_became_peered":"2024-03-18T15:34:22.785695+0000","last_unstale":"2024-03-18T15:34:23.744057+0000","last_undegraded":"2024-03-18T15:34:23.744057+0000","last_fullsized":"2024-03-18T15:34:23.744057+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:51:42.881058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.744019+0000","last_change":"2024-03-18T15:34:22.758732+0000","last_active":"2024-03-18T15:34:23.744019+0000","last_peered":"2024-03-18T15:34:23.744019+0000","last_clean":"2024-03-18T15:34:23.744019+0000","last_became_active":"2024-03-18T15:34:22.758033+0000","last_became_peered":"2024-03-18T15:34:22.758033+0000","last_unstale":"2024-03-18T15:34:23.744019+0000","last_undegraded":"2024-03-18T15:34:23.744019+0000","last_fullsized":"2024-03-18T15:34:23.744019+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:49:06.417569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.746246+0000","last_change":"2024-03-18T15:34:22.791072+0000","last_active":"2024-03-18T15:34:23.746246+0000","last_peered":"2024-03-18T15:34:23.746246+0000","last_clean":"2024-03-18T15:34:23.746246+0000","last_became_active":"2024-03-18T15:34:22.790267+0000","last_became_peered":"2024-03-18T15:34:22.790267+0000","last_unstale":"2024-03-18T15:34:23.746246+0000","last_undegraded":"2024-03-18T15:34:23.746246+0000","last_fullsized":"2024-03-18T15:34:23.746246+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:03:05.969919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.751799+0000","last_change":"2024-03-18T15:34:22.773037+0000","last_active":"2024-03-18T15:34:23.751799+0000","last_peered":"2024-03-18T15:34:23.751799+0000","last_clean":"2024-03-18T15:34:23.751799+0000","last_became_active":"2024-03-18T15:34:22.772714+0000","last_became_peered":"2024-03-18T15:34:22.772714+0000","last_unstale":"2024-03-18T15:34:23.751799+0000","last_undegraded":"2024-03-18T15:34:23.751799+0000","last_fullsized":"2024-03-18T15:34:23.751799+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:17:02.398404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.744088+0000","last_change":"2024-03-18T15:34:22.763800+0000","last_active":"2024-03-18T15:34:23.744088+0000","last_peered":"2024-03-18T15:34:23.744088+0000","last_clean":"2024-03-18T15:34:23.744088+0000","last_became_active":"2024-03-18T15:34:22.762781+0000","last_became_peered":"2024-03-18T15:34:22.762781+0000","last_unstale":"2024-03-18T15:34:23.744088+0000","last_undegraded":"2024-03-18T15:34:23.744088+0000","last_fullsized":"2024-03-18T15:34:23.744088+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:36:43.593724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-18T15:34:21.723958+0000","last_change":"2024-03-18T15:34:21.723958+0000","last_active":"2024-03-18T15:34:21.723958+0000","last_peered":"2024-03-18T15:34:21.723958+0000","last_clean":"2024-03-18T15:34:21.723958+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-18T15:34:21.723958+0000","last_undegraded":"2024-03-18T15:34:21.723958+0000","last_fullsized":"2024-03-18T15:34:21.723958+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.742897+0000","last_change":"2024-03-18T15:34:22.751395+0000","last_active":"2024-03-18T15:34:23.742897+0000","last_peered":"2024-03-18T15:34:23.742897+0000","last_clean":"2024-03-18T15:34:23.742897+0000","last_became_active":"2024-03-18T15:34:22.751047+0000","last_became_peered":"2024-03-18T15:34:22.751047+0000","last_unstale":"2024-03-18T15:34:23.742897+0000","last_undegraded":"2024-03-18T15:34:23.742897+0000","last_fullsized":"2024-03-18T15:34:23.742897+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:31:27.596543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-18T15:34:22.781158+0000","last_change":"2024-03-18T15:34:22.767530+0000","last_active":"2024-03-18T15:34:22.781158+0000","last_peered":"2024-03-18T15:34:22.781158+0000","last_clean":"2024-03-18T15:34:22.781158+0000","last_became_active":"2024-03-18T15:34:22.762123+0000","last_became_peered":"2024-03-18T15:34:22.762123+0000","last_unstale":"2024-03-18T15:34:22.781158+0000","last_undegraded":"2024-03-18T15:34:22.781158+0000","last_fullsized":"2024-03-18T15:34:22.781158+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:29:09.934918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-18T15:34:21.723958+0000","last_change":"2024-03-18T15:34:21.723958+0000","last_active":"2024-03-18T15:34:21.723958+0000","last_peered":"2024-03-18T15:34:21.723958+0000","last_clean":"2024-03-18T15:34:21.723958+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-18T15:34:21.723958+0000","last_undegraded":"2024-03-18T15:34:21.723958+0000","last_fullsized":"2024-03-18T15:34:21.723958+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.745086+0000","last_change":"2024-03-18T15:34:22.768015+0000","last_active":"2024-03-18T15:34:23.745086+0000","last_peered":"2024-03-18T15:34:23.745086+0000","last_clean":"2024-03-18T15:34:23.745086+0000","last_became_active":"2024-03-18T15:34:22.767310+0000","last_became_peered":"2024-03-18T15:34:22.767310+0000","last_unstale":"2024-03-18T15:34:23.745086+0000","last_undegraded":"2024-03-18T15:34:23.745086+0000","last_fullsized":"2024-03-18T15:34:23.745086+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:18:33.478792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.744439+0000","last_change":"2024-03-18T15:34:22.790197+0000","last_active":"2024-03-18T15:34:23.744439+0000","last_peered":"2024-03-18T15:34:23.744439+0000","last_clean":"2024-03-18T15:34:23.744439+0000","last_became_active":"2024-03-18T15:34:22.789575+0000","last_became_peered":"2024-03-18T15:34:22.789575+0000","last_unstale":"2024-03-18T15:34:23.744439+0000","last_undegraded":"2024-03-18T15:34:23.744439+0000","last_fullsized":"2024-03-18T15:34:23.744439+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:25:26.289800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-18T15:34:22.783145+0000","last_change":"2024-03-18T15:34:22.775054+0000","last_active":"2024-03-18T15:34:22.783145+0000","last_peered":"2024-03-18T15:34:22.783145+0000","last_clean":"2024-03-18T15:34:22.783145+0000","last_became_active":"2024-03-18T15:34:22.772194+0000","last_became_peered":"2024-03-18T15:34:22.772194+0000","last_unstale":"2024-03-18T15:34:22.783145+0000","last_undegraded":"2024-03-18T15:34:22.783145+0000","last_fullsized":"2024-03-18T15:34:22.783145+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:16:42.700006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-18T15:34:22.783079+0000","last_change":"2024-03-18T15:34:22.772172+0000","last_active":"2024-03-18T15:34:22.783079+0000","last_peered":"2024-03-18T15:34:22.783079+0000","last_clean":"2024-03-18T15:34:22.783079+0000","last_became_active":"2024-03-18T15:34:22.771755+0000","last_became_peered":"2024-03-18T15:34:22.771755+0000","last_unstale":"2024-03-18T15:34:22.783079+0000","last_undegraded":"2024-03-18T15:34:22.783079+0000","last_fullsized":"2024-03-18T15:34:22.783079+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:57:00.519939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.743530+0000","last_change":"2024-03-18T15:34:22.780542+0000","last_active":"2024-03-18T15:34:23.743530+0000","last_peered":"2024-03-18T15:34:23.743530+0000","last_clean":"2024-03-18T15:34:23.743530+0000","last_became_active":"2024-03-18T15:34:22.780261+0000","last_became_peered":"2024-03-18T15:34:22.780261+0000","last_unstale":"2024-03-18T15:34:23.743530+0000","last_undegraded":"2024-03-18T15:34:23.743530+0000","last_fullsized":"2024-03-18T15:34:23.743530+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:24:52.230317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.744707+0000","last_change":"2024-03-18T15:34:22.772477+0000","last_active":"2024-03-18T15:34:23.744707+0000","last_peered":"2024-03-18T15:34:23.744707+0000","last_clean":"2024-03-18T15:34:23.744707+0000","last_became_active":"2024-03-18T15:34:22.771735+0000","last_became_peered":"2024-03-18T15:34:22.771735+0000","last_unstale":"2024-03-18T15:34:23.744707+0000","last_undegraded":"2024-03-18T15:34:23.744707+0000","last_fullsized":"2024-03-18T15:34:23.744707+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:52:34.297832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.744641+0000","last_change":"2024-03-18T15:34:22.763214+0000","last_active":"2024-03-18T15:34:23.744641+0000","last_peered":"2024-03-18T15:34:23.744641+0000","last_clean":"2024-03-18T15:34:23.744641+0000","last_became_active":"2024-03-18T15:34:22.762356+0000","last_became_peered":"2024-03-18T15:34:22.762356+0000","last_unstale":"2024-03-18T15:34:23.744641+0000","last_undegraded":"2024-03-18T15:34:23.744641+0000","last_fullsized":"2024-03-18T15:34:23.744641+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:43:37.362381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.749211+0000","last_change":"2024-03-18T15:34:22.770613+0000","last_active":"2024-03-18T15:34:23.749211+0000","last_peered":"2024-03-18T15:34:23.749211+0000","last_clean":"2024-03-18T15:34:23.749211+0000","last_became_active":"2024-03-18T15:34:22.770340+0000","last_became_peered":"2024-03-18T15:34:22.770340+0000","last_unstale":"2024-03-18T15:34:23.749211+0000","last_undegraded":"2024-03-18T15:34:23.749211+0000","last_fullsized":"2024-03-18T15:34:23.749211+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:01:17.226430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-18T15:34:21.723958+0000","last_change":"2024-03-18T15:34:21.723958+0000","last_active":"2024-03-18T15:34:21.723958+0000","last_peered":"2024-03-18T15:34:21.723958+0000","last_clean":"2024-03-18T15:34:21.723958+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-18T15:34:21.723958+0000","last_undegraded":"2024-03-18T15:34:21.723958+0000","last_fullsized":"2024-03-18T15:34:21.723958+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.752145+0000","last_change":"2024-03-18T15:34:22.766888+0000","last_active":"2024-03-18T15:34:23.752145+0000","last_peered":"2024-03-18T15:34:23.752145+0000","last_clean":"2024-03-18T15:34:23.752145+0000","last_became_active":"2024-03-18T15:34:22.766610+0000","last_became_peered":"2024-03-18T15:34:22.766610+0000","last_unstale":"2024-03-18T15:34:23.752145+0000","last_undegraded":"2024-03-18T15:34:23.752145+0000","last_fullsized":"2024-03-18T15:34:23.752145+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:02:32.023342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-18T15:34:22.783440+0000","last_change":"2024-03-18T15:34:22.776017+0000","last_active":"2024-03-18T15:34:22.783440+0000","last_peered":"2024-03-18T15:34:22.783440+0000","last_clean":"2024-03-18T15:34:22.783440+0000","last_became_active":"2024-03-18T15:34:22.771540+0000","last_became_peered":"2024-03-18T15:34:22.771540+0000","last_unstale":"2024-03-18T15:34:22.783440+0000","last_undegraded":"2024-03-18T15:34:22.783440+0000","last_fullsized":"2024-03-18T15:34:22.783440+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:58:39.514486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.752850+0000","last_change":"2024-03-18T15:34:22.779619+0000","last_active":"2024-03-18T15:34:23.752850+0000","last_peered":"2024-03-18T15:34:23.752850+0000","last_clean":"2024-03-18T15:34:23.752850+0000","last_became_active":"2024-03-18T15:34:22.779331+0000","last_became_peered":"2024-03-18T15:34:22.779331+0000","last_unstale":"2024-03-18T15:34:23.752850+0000","last_undegraded":"2024-03-18T15:34:23.752850+0000","last_fullsized":"2024-03-18T15:34:23.752850+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:25:38.946261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-18T15:34:21.723958+0000","last_change":"2024-03-18T15:34:21.723958+0000","last_active":"2024-03-18T15:34:21.723958+0000","last_peered":"2024-03-18T15:34:21.723958+0000","last_clean":"2024-03-18T15:34:21.723958+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-18T15:34:21.723958+0000","last_undegraded":"2024-03-18T15:34:21.723958+0000","last_fullsized":"2024-03-18T15:34:21.723958+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.745049+0000","last_change":"2024-03-18T15:34:22.766061+0000","last_active":"2024-03-18T15:34:23.745049+0000","last_peered":"2024-03-18T15:34:23.745049+0000","last_clean":"2024-03-18T15:34:23.745049+0000","last_became_active":"2024-03-18T15:34:22.764782+0000","last_became_peered":"2024-03-18T15:34:22.764782+0000","last_unstale":"2024-03-18T15:34:23.745049+0000","last_undegraded":"2024-03-18T15:34:23.745049+0000","last_fullsized":"2024-03-18T15:34:23.745049+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:29:09.934918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.744517+0000","last_change":"2024-03-18T15:34:22.785047+0000","last_active":"2024-03-18T15:34:23.744517+0000","last_peered":"2024-03-18T15:34:23.744517+0000","last_clean":"2024-03-18T15:34:23.744517+0000","last_became_active":"2024-03-18T15:34:22.784398+0000","last_became_peered":"2024-03-18T15:34:22.784398+0000","last_unstale":"2024-03-18T15:34:23.744517+0000","last_undegraded":"2024-03-18T15:34:23.744517+0000","last_fullsized":"2024-03-18T15:34:23.744517+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:20:45.678969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-18T15:34:21.723958+0000","last_change":"2024-03-18T15:34:21.723958+0000","last_active":"2024-03-18T15:34:21.723958+0000","last_peered":"2024-03-18T15:34:21.723958+0000","last_clean":"2024-03-18T15:34:21.723958+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-18T15:34:21.723958+0000","last_undegraded":"2024-03-18T15:34:21.723958+0000","last_fullsized":"2024-03-18T15:34:21.723958+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.744525+0000","last_change":"2024-03-18T15:34:22.776424+0000","last_active":"2024-03-18T15:34:23.744525+0000","last_peered":"2024-03-18T15:34:23.744525+0000","last_clean":"2024-03-18T15:34:23.744525+0000","last_became_active":"2024-03-18T15:34:22.775691+0000","last_became_peered":"2024-03-18T15:34:22.775691+0000","last_unstale":"2024-03-18T15:34:23.744525+0000","last_undegraded":"2024-03-18T15:34:23.744525+0000","last_fullsized":"2024-03-18T15:34:23.744525+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:57:53.266016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.749412+0000","last_change":"2024-03-18T15:34:22.777517+0000","last_active":"2024-03-18T15:34:23.749412+0000","last_peered":"2024-03-18T15:34:23.749412+0000","last_clean":"2024-03-18T15:34:23.749412+0000","last_became_active":"2024-03-18T15:34:22.777084+0000","last_became_peered":"2024-03-18T15:34:22.777084+0000","last_unstale":"2024-03-18T15:34:23.749412+0000","last_undegraded":"2024-03-18T15:34:23.749412+0000","last_fullsized":"2024-03-18T15:34:23.749412+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:19:13.472867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-18T15:34:20.713943+0000","last_change":"2024-03-18T15:34:20.713943+0000","last_active":"2024-03-18T15:34:20.713943+0000","last_peered":"2024-03-18T15:34:20.713943+0000","last_clean":"2024-03-18T15:34:20.713943+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-18T15:34:20.713943+0000","last_undegraded":"2024-03-18T15:34:20.713943+0000","last_fullsized":"2024-03-18T15:34:20.713943+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-18T15:34:21.723958+0000","last_change":"2024-03-18T15:34:21.723958+0000","last_active":"2024-03-18T15:34:21.723958+0000","last_peered":"2024-03-18T15:34:21.723958+0000","last_clean":"2024-03-18T15:34:21.723958+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-18T15:34:21.723958+0000","last_undegraded":"2024-03-18T15:34:21.723958+0000","last_fullsized":"2024-03-18T15:34:21.723958+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"29'1","reported_seq":15,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.749280+0000","last_change":"2024-03-18T15:34:21.771464+0000","last_active":"2024-03-18T15:34:23.749280+0000","last_peered":"2024-03-18T15:34:23.749280+0000","last_clean":"2024-03-18T15:34:23.749280+0000","last_became_active":"2024-03-18T15:34:21.771234+0000","last_became_peered":"2024-03-18T15:34:21.771234+0000","last_unstale":"2024-03-18T15:34:23.749280+0000","last_undegraded":"2024-03-18T15:34:23.749280+0000","last_fullsized":"2024-03-18T15:34:23.749280+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T15:57:42.258826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.745934+0000","last_change":"2024-03-18T15:34:22.785781+0000","last_active":"2024-03-18T15:34:23.745934+0000","last_peered":"2024-03-18T15:34:23.745934+0000","last_clean":"2024-03-18T15:34:23.745934+0000","last_became_active":"2024-03-18T15:34:22.785119+0000","last_became_peered":"2024-03-18T15:34:22.785119+0000","last_unstale":"2024-03-18T15:34:23.745934+0000","last_undegraded":"2024-03-18T15:34:23.745934+0000","last_fullsized":"2024-03-18T15:34:23.745934+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:57:00.519939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"29'2","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-18T15:34:22.775760+0000","last_change":"2024-03-18T15:34:21.770272+0000","last_active":"2024-03-18T15:34:22.775760+0000","last_peered":"2024-03-18T15:34:22.775760+0000","last_clean":"2024-03-18T15:34:22.775760+0000","last_became_active":"2024-03-18T15:34:21.769876+0000","last_became_peered":"2024-03-18T15:34:21.769876+0000","last_unstale":"2024-03-18T15:34:22.775760+0000","last_undegraded":"2024-03-18T15:34:22.775760+0000","last_fullsized":"2024-03-18T15:34:22.775760+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:09:09.582374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.752782+0000","last_change":"2024-03-18T15:34:22.779629+0000","last_active":"2024-03-18T15:34:23.752782+0000","last_peered":"2024-03-18T15:34:23.752782+0000","last_clean":"2024-03-18T15:34:23.752782+0000","last_became_active":"2024-03-18T15:34:22.778145+0000","last_became_peered":"2024-03-18T15:34:22.778145+0000","last_unstale":"2024-03-18T15:34:23.752782+0000","last_undegraded":"2024-03-18T15:34:23.752782+0000","last_fullsized":"2024-03-18T15:34:23.752782+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:13:16.770552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"29'2","reported_seq":16,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.743679+0000","last_change":"2024-03-18T15:34:21.765588+0000","last_active":"2024-03-18T15:34:23.743679+0000","last_peered":"2024-03-18T15:34:23.743679+0000","last_clean":"2024-03-18T15:34:23.743679+0000","last_became_active":"2024-03-18T15:34:21.764531+0000","last_became_peered":"2024-03-18T15:34:21.764531+0000","last_unstale":"2024-03-18T15:34:23.743679+0000","last_undegraded":"2024-03-18T15:34:23.743679+0000","last_fullsized":"2024-03-18T15:34:23.743679+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:59:41.899954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.744918+0000","last_change":"2024-03-18T15:34:22.792570+0000","last_active":"2024-03-18T15:34:23.744918+0000","last_peered":"2024-03-18T15:34:23.744918+0000","last_clean":"2024-03-18T15:34:23.744918+0000","last_became_active":"2024-03-18T15:34:22.792028+0000","last_became_peered":"2024-03-18T15:34:22.792028+0000","last_unstale":"2024-03-18T15:34:23.744918+0000","last_undegraded":"2024-03-18T15:34:23.744918+0000","last_fullsized":"2024-03-18T15:34:23.744918+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:39:27.621959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"29'1","reported_seq":15,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.745361+0000","last_change":"2024-03-18T15:34:21.754670+0000","last_active":"2024-03-18T15:34:23.745361+0000","last_peered":"2024-03-18T15:34:23.745361+0000","last_clean":"2024-03-18T15:34:23.745361+0000","last_became_active":"2024-03-18T15:34:21.753341+0000","last_became_peered":"2024-03-18T15:34:21.753341+0000","last_unstale":"2024-03-18T15:34:23.745361+0000","last_undegraded":"2024-03-18T15:34:23.745361+0000","last_fullsized":"2024-03-18T15:34:23.745361+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:03:23.511853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-18T15:34:21.723958+0000","last_change":"2024-03-18T15:34:21.723958+0000","last_active":"2024-03-18T15:34:21.723958+0000","last_peered":"2024-03-18T15:34:21.723958+0000","last_clean":"2024-03-18T15:34:21.723958+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-18T15:34:21.723958+0000","last_undegraded":"2024-03-18T15:34:21.723958+0000","last_fullsized":"2024-03-18T15:34:21.723958+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"29'1","reported_seq":15,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.744929+0000","last_change":"2024-03-18T15:34:21.754688+0000","last_active":"2024-03-18T15:34:23.744929+0000","last_peered":"2024-03-18T15:34:23.744929+0000","last_clean":"2024-03-18T15:34:23.744929+0000","last_became_active":"2024-03-18T15:34:21.753425+0000","last_became_peered":"2024-03-18T15:34:21.753425+0000","last_unstale":"2024-03-18T15:34:23.744929+0000","last_undegraded":"2024-03-18T15:34:23.744929+0000","last_fullsized":"2024-03-18T15:34:23.744929+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:53:25.674607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.742773+0000","last_change":"2024-03-18T15:34:22.770699+0000","last_active":"2024-03-18T15:34:23.742773+0000","last_peered":"2024-03-18T15:34:23.742773+0000","last_clean":"2024-03-18T15:34:23.742773+0000","last_became_active":"2024-03-18T15:34:22.769967+0000","last_became_peered":"2024-03-18T15:34:22.769967+0000","last_unstale":"2024-03-18T15:34:23.742773+0000","last_undegraded":"2024-03-18T15:34:23.742773+0000","last_fullsized":"2024-03-18T15:34:23.742773+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:22:56.512250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-18T15:34:22.756784+0000","last_change":"2024-03-18T15:34:21.761313+0000","last_active":"2024-03-18T15:34:22.756784+0000","last_peered":"2024-03-18T15:34:22.756784+0000","last_clean":"2024-03-18T15:34:22.756784+0000","last_became_active":"2024-03-18T15:34:21.760662+0000","last_became_peered":"2024-03-18T15:34:21.760662+0000","last_unstale":"2024-03-18T15:34:22.756784+0000","last_undegraded":"2024-03-18T15:34:22.756784+0000","last_fullsized":"2024-03-18T15:34:22.756784+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:44:05.246702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.751740+0000","last_change":"2024-03-18T15:34:22.768121+0000","last_active":"2024-03-18T15:34:23.751740+0000","last_peered":"2024-03-18T15:34:23.751740+0000","last_clean":"2024-03-18T15:34:23.751740+0000","last_became_active":"2024-03-18T15:34:22.767794+0000","last_became_peered":"2024-03-18T15:34:22.767794+0000","last_unstale":"2024-03-18T15:34:23.751740+0000","last_undegraded":"2024-03-18T15:34:23.751740+0000","last_fullsized":"2024-03-18T15:34:23.751740+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:03:05.969919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.742642+0000","last_change":"2024-03-18T15:34:21.769291+0000","last_active":"2024-03-18T15:34:23.742642+0000","last_peered":"2024-03-18T15:34:23.742642+0000","last_clean":"2024-03-18T15:34:23.742642+0000","last_became_active":"2024-03-18T15:34:21.768605+0000","last_became_peered":"2024-03-18T15:34:21.768605+0000","last_unstale":"2024-03-18T15:34:23.742642+0000","last_undegraded":"2024-03-18T15:34:23.742642+0000","last_fullsized":"2024-03-18T15:34:23.742642+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:05:37.023288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-18T15:34:21.723958+0000","last_change":"2024-03-18T15:34:21.723958+0000","last_active":"2024-03-18T15:34:21.723958+0000","last_peered":"2024-03-18T15:34:21.723958+0000","last_clean":"2024-03-18T15:34:21.723958+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-18T15:34:21.723958+0000","last_undegraded":"2024-03-18T15:34:21.723958+0000","last_fullsized":"2024-03-18T15:34:21.723958+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-18T15:34:22.756998+0000","last_change":"2024-03-18T15:34:21.760191+0000","last_active":"2024-03-18T15:34:22.756998+0000","last_peered":"2024-03-18T15:34:22.756998+0000","last_clean":"2024-03-18T15:34:22.756998+0000","last_became_active":"2024-03-18T15:34:21.759764+0000","last_became_peered":"2024-03-18T15:34:21.759764+0000","last_unstale":"2024-03-18T15:34:22.756998+0000","last_undegraded":"2024-03-18T15:34:22.756998+0000","last_fullsized":"2024-03-18T15:34:22.756998+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:46:06.301575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-18T15:34:22.783124+0000","last_change":"2024-03-18T15:34:22.772207+0000","last_active":"2024-03-18T15:34:22.783124+0000","last_peered":"2024-03-18T15:34:22.783124+0000","last_clean":"2024-03-18T15:34:22.783124+0000","last_became_active":"2024-03-18T15:34:22.771830+0000","last_became_peered":"2024-03-18T15:34:22.771830+0000","last_unstale":"2024-03-18T15:34:22.783124+0000","last_undegraded":"2024-03-18T15:34:22.783124+0000","last_fullsized":"2024-03-18T15:34:22.783124+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:03:05.969919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.749382+0000","last_change":"2024-03-18T15:34:21.758575+0000","last_active":"2024-03-18T15:34:23.749382+0000","last_peered":"2024-03-18T15:34:23.749382+0000","last_clean":"2024-03-18T15:34:23.749382+0000","last_became_active":"2024-03-18T15:34:21.758127+0000","last_became_peered":"2024-03-18T15:34:21.758127+0000","last_unstale":"2024-03-18T15:34:23.749382+0000","last_undegraded":"2024-03-18T15:34:23.749382+0000","last_fullsized":"2024-03-18T15:34:23.749382+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:03:37.411256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-18T15:34:22.783527+0000","last_change":"2024-03-18T15:34:22.776364+0000","last_active":"2024-03-18T15:34:22.783527+0000","last_peered":"2024-03-18T15:34:22.783527+0000","last_clean":"2024-03-18T15:34:22.783527+0000","last_became_active":"2024-03-18T15:34:22.775298+0000","last_became_peered":"2024-03-18T15:34:22.775298+0000","last_unstale":"2024-03-18T15:34:22.783527+0000","last_undegraded":"2024-03-18T15:34:22.783527+0000","last_fullsized":"2024-03-18T15:34:22.783527+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:13:16.770552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-18T15:34:20.713943+0000","last_change":"2024-03-18T15:34:20.713943+0000","last_active":"2024-03-18T15:34:20.713943+0000","last_peered":"2024-03-18T15:34:20.713943+0000","last_clean":"2024-03-18T15:34:20.713943+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-18T15:34:20.713943+0000","last_undegraded":"2024-03-18T15:34:20.713943+0000","last_fullsized":"2024-03-18T15:34:20.713943+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-18T15:34:21.723958+0000","last_change":"2024-03-18T15:34:21.723958+0000","last_active":"2024-03-18T15:34:21.723958+0000","last_peered":"2024-03-18T15:34:21.723958+0000","last_clean":"2024-03-18T15:34:21.723958+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-18T15:34:21.723958+0000","last_undegraded":"2024-03-18T15:34:21.723958+0000","last_fullsized":"2024-03-18T15:34:21.723958+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"29'6","reported_seq":18,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-18T15:34:22.794071+0000","last_change":"2024-03-18T15:34:21.765099+0000","last_active":"2024-03-18T15:34:22.794071+0000","last_peered":"2024-03-18T15:34:22.794071+0000","last_clean":"2024-03-18T15:34:22.794071+0000","last_became_active":"2024-03-18T15:34:21.764694+0000","last_became_peered":"2024-03-18T15:34:21.764694+0000","last_unstale":"2024-03-18T15:34:22.794071+0000","last_undegraded":"2024-03-18T15:34:22.794071+0000","last_fullsized":"2024-03-18T15:34:22.794071+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:22:55.502235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.751476+0000","last_change":"2024-03-18T15:34:22.779939+0000","last_active":"2024-03-18T15:34:23.751476+0000","last_peered":"2024-03-18T15:34:23.751476+0000","last_clean":"2024-03-18T15:34:23.751476+0000","last_became_active":"2024-03-18T15:34:22.778840+0000","last_became_peered":"2024-03-18T15:34:22.778840+0000","last_unstale":"2024-03-18T15:34:23.751476+0000","last_undegraded":"2024-03-18T15:34:23.751476+0000","last_fullsized":"2024-03-18T15:34:23.751476+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:57:00.519939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"29'2","reported_seq":16,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.745449+0000","last_change":"2024-03-18T15:34:21.769150+0000","last_active":"2024-03-18T15:34:23.745449+0000","last_peered":"2024-03-18T15:34:23.745449+0000","last_clean":"2024-03-18T15:34:23.745449+0000","last_became_active":"2024-03-18T15:34:21.768408+0000","last_became_peered":"2024-03-18T15:34:21.768408+0000","last_unstale":"2024-03-18T15:34:23.745449+0000","last_undegraded":"2024-03-18T15:34:23.745449+0000","last_fullsized":"2024-03-18T15:34:23.745449+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:55:30.386254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-18T15:34:22.783216+0000","last_change":"2024-03-18T15:34:22.776102+0000","last_active":"2024-03-18T15:34:22.783216+0000","last_peered":"2024-03-18T15:34:22.783216+0000","last_clean":"2024-03-18T15:34:22.783216+0000","last_became_active":"2024-03-18T15:34:22.768547+0000","last_became_peered":"2024-03-18T15:34:22.768547+0000","last_unstale":"2024-03-18T15:34:22.783216+0000","last_undegraded":"2024-03-18T15:34:22.783216+0000","last_fullsized":"2024-03-18T15:34:22.783216+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:02:32.023342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.751528+0000","last_change":"2024-03-18T15:34:21.762710+0000","last_active":"2024-03-18T15:34:23.751528+0000","last_peered":"2024-03-18T15:34:23.751528+0000","last_clean":"2024-03-18T15:34:23.751528+0000","last_became_active":"2024-03-18T15:34:21.762295+0000","last_became_peered":"2024-03-18T15:34:21.762295+0000","last_unstale":"2024-03-18T15:34:23.751528+0000","last_undegraded":"2024-03-18T15:34:23.751528+0000","last_fullsized":"2024-03-18T15:34:23.751528+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:55:00.926699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.744897+0000","last_change":"2024-03-18T15:34:22.792550+0000","last_active":"2024-03-18T15:34:23.744897+0000","last_peered":"2024-03-18T15:34:23.744897+0000","last_clean":"2024-03-18T15:34:23.744897+0000","last_became_active":"2024-03-18T15:34:22.791889+0000","last_became_peered":"2024-03-18T15:34:22.791889+0000","last_unstale":"2024-03-18T15:34:23.744897+0000","last_undegraded":"2024-03-18T15:34:23.744897+0000","last_fullsized":"2024-03-18T15:34:23.744897+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:11:46.630503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-18T15:34:22.757106+0000","last_change":"2024-03-18T15:34:21.771889+0000","last_active":"2024-03-18T15:34:22.757106+0000","last_peered":"2024-03-18T15:34:22.757106+0000","last_clean":"2024-03-18T15:34:22.757106+0000","last_became_active":"2024-03-18T15:34:21.771441+0000","last_became_peered":"2024-03-18T15:34:21.771441+0000","last_unstale":"2024-03-18T15:34:22.757106+0000","last_undegraded":"2024-03-18T15:34:22.757106+0000","last_fullsized":"2024-03-18T15:34:22.757106+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:43:36.352366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-18T15:34:21.723958+0000","last_change":"2024-03-18T15:34:21.723958+0000","last_active":"2024-03-18T15:34:21.723958+0000","last_peered":"2024-03-18T15:34:21.723958+0000","last_clean":"2024-03-18T15:34:21.723958+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-18T15:34:21.723958+0000","last_undegraded":"2024-03-18T15:34:21.723958+0000","last_fullsized":"2024-03-18T15:34:21.723958+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"29'8","reported_seq":20,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-18T15:34:22.783177+0000","last_change":"2024-03-18T15:34:21.764809+0000","last_active":"2024-03-18T15:34:22.783177+0000","last_peered":"2024-03-18T15:34:22.783177+0000","last_clean":"2024-03-18T15:34:22.783177+0000","last_became_active":"2024-03-18T15:34:21.764537+0000","last_became_peered":"2024-03-18T15:34:21.764537+0000","last_unstale":"2024-03-18T15:34:22.783177+0000","last_undegraded":"2024-03-18T15:34:22.783177+0000","last_fullsized":"2024-03-18T15:34:22.783177+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:49:56.976995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.745893+0000","last_change":"2024-03-18T15:34:22.777789+0000","last_active":"2024-03-18T15:34:23.745893+0000","last_peered":"2024-03-18T15:34:23.745893+0000","last_clean":"2024-03-18T15:34:23.745893+0000","last_became_active":"2024-03-18T15:34:22.777052+0000","last_became_peered":"2024-03-18T15:34:22.777052+0000","last_unstale":"2024-03-18T15:34:23.745893+0000","last_undegraded":"2024-03-18T15:34:23.745893+0000","last_fullsized":"2024-03-18T15:34:23.745893+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:21:26.063255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"29'1","reported_seq":15,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.752480+0000","last_change":"2024-03-18T15:34:21.771611+0000","last_active":"2024-03-18T15:34:23.752480+0000","last_peered":"2024-03-18T15:34:23.752480+0000","last_clean":"2024-03-18T15:34:23.752480+0000","last_became_active":"2024-03-18T15:34:21.769590+0000","last_became_peered":"2024-03-18T15:34:21.769590+0000","last_unstale":"2024-03-18T15:34:23.752480+0000","last_undegraded":"2024-03-18T15:34:23.752480+0000","last_fullsized":"2024-03-18T15:34:23.752480+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:34:03.948098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-18T15:34:21.723958+0000","last_change":"2024-03-18T15:34:21.723958+0000","last_active":"2024-03-18T15:34:21.723958+0000","last_peered":"2024-03-18T15:34:21.723958+0000","last_clean":"2024-03-18T15:34:21.723958+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-18T15:34:21.723958+0000","last_undegraded":"2024-03-18T15:34:21.723958+0000","last_fullsized":"2024-03-18T15:34:21.723958+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"29'2","reported_seq":16,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.745821+0000","last_change":"2024-03-18T15:34:21.769268+0000","last_active":"2024-03-18T15:34:23.745821+0000","last_peered":"2024-03-18T15:34:23.745821+0000","last_clean":"2024-03-18T15:34:23.745821+0000","last_became_active":"2024-03-18T15:34:21.768518+0000","last_became_peered":"2024-03-18T15:34:21.768518+0000","last_unstale":"2024-03-18T15:34:23.745821+0000","last_undegraded":"2024-03-18T15:34:23.745821+0000","last_fullsized":"2024-03-18T15:34:23.745821+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:53:02.656952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.752032+0000","last_change":"2024-03-18T15:34:22.777582+0000","last_active":"2024-03-18T15:34:23.752032+0000","last_peered":"2024-03-18T15:34:23.752032+0000","last_clean":"2024-03-18T15:34:23.752032+0000","last_became_active":"2024-03-18T15:34:22.777227+0000","last_became_peered":"2024-03-18T15:34:22.777227+0000","last_unstale":"2024-03-18T15:34:23.752032+0000","last_undegraded":"2024-03-18T15:34:23.752032+0000","last_fullsized":"2024-03-18T15:34:23.752032+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:58:39.514486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"29'1","reported_seq":15,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.746021+0000","last_change":"2024-03-18T15:34:21.758424+0000","last_active":"2024-03-18T15:34:23.746021+0000","last_peered":"2024-03-18T15:34:23.746021+0000","last_clean":"2024-03-18T15:34:23.746021+0000","last_became_active":"2024-03-18T15:34:21.756927+0000","last_became_peered":"2024-03-18T15:34:21.756927+0000","last_unstale":"2024-03-18T15:34:23.746021+0000","last_undegraded":"2024-03-18T15:34:23.746021+0000","last_fullsized":"2024-03-18T15:34:23.746021+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:20:41.054645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.743489+0000","last_change":"2024-03-18T15:34:22.764733+0000","last_active":"2024-03-18T15:34:23.743489+0000","last_peered":"2024-03-18T15:34:23.743489+0000","last_clean":"2024-03-18T15:34:23.743489+0000","last_became_active":"2024-03-18T15:34:22.764348+0000","last_became_peered":"2024-03-18T15:34:22.764348+0000","last_unstale":"2024-03-18T15:34:23.743489+0000","last_undegraded":"2024-03-18T15:34:23.743489+0000","last_fullsized":"2024-03-18T15:34:23.743489+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:23:53.146495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.745767+0000","last_change":"2024-03-18T15:34:21.755902+0000","last_active":"2024-03-18T15:34:23.745767+0000","last_peered":"2024-03-18T15:34:23.745767+0000","last_clean":"2024-03-18T15:34:23.745767+0000","last_became_active":"2024-03-18T15:34:21.755004+0000","last_became_peered":"2024-03-18T15:34:21.755004+0000","last_unstale":"2024-03-18T15:34:23.745767+0000","last_undegraded":"2024-03-18T15:34:23.745767+0000","last_fullsized":"2024-03-18T15:34:23.745767+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:57:48.185270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.742694+0000","last_change":"2024-03-18T15:34:22.793199+0000","last_active":"2024-03-18T15:34:23.742694+0000","last_peered":"2024-03-18T15:34:23.742694+0000","last_clean":"2024-03-18T15:34:23.742694+0000","last_became_active":"2024-03-18T15:34:22.792851+0000","last_became_peered":"2024-03-18T15:34:22.792851+0000","last_unstale":"2024-03-18T15:34:23.742694+0000","last_undegraded":"2024-03-18T15:34:23.742694+0000","last_fullsized":"2024-03-18T15:34:23.742694+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:25:24.865468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"29'2","reported_seq":16,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.749244+0000","last_change":"2024-03-18T15:34:21.769215+0000","last_active":"2024-03-18T15:34:23.749244+0000","last_peered":"2024-03-18T15:34:23.749244+0000","last_clean":"2024-03-18T15:34:23.749244+0000","last_became_active":"2024-03-18T15:34:21.768612+0000","last_became_peered":"2024-03-18T15:34:21.768612+0000","last_unstale":"2024-03-18T15:34:23.749244+0000","last_undegraded":"2024-03-18T15:34:23.749244+0000","last_fullsized":"2024-03-18T15:34:23.749244+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:58:53.162823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.743038+0000","last_change":"2024-03-18T15:34:22.768133+0000","last_active":"2024-03-18T15:34:23.743038+0000","last_peered":"2024-03-18T15:34:23.743038+0000","last_clean":"2024-03-18T15:34:23.743038+0000","last_became_active":"2024-03-18T15:34:22.767842+0000","last_became_peered":"2024-03-18T15:34:22.767842+0000","last_unstale":"2024-03-18T15:34:23.743038+0000","last_undegraded":"2024-03-18T15:34:23.743038+0000","last_fullsized":"2024-03-18T15:34:23.743038+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:17:46.800698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-18T15:34:20.713943+0000","last_change":"2024-03-18T15:34:20.713943+0000","last_active":"2024-03-18T15:34:20.713943+0000","last_peered":"2024-03-18T15:34:20.713943+0000","last_clean":"2024-03-18T15:34:20.713943+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-18T15:34:20.713943+0000","last_undegraded":"2024-03-18T15:34:20.713943+0000","last_fullsized":"2024-03-18T15:34:20.713943+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.742435+0000","last_change":"2024-03-18T15:34:22.752889+0000","last_active":"2024-03-18T15:34:23.742435+0000","last_peered":"2024-03-18T15:34:23.742435+0000","last_clean":"2024-03-18T15:34:23.742435+0000","last_became_active":"2024-03-18T15:34:22.752637+0000","last_became_peered":"2024-03-18T15:34:22.752637+0000","last_unstale":"2024-03-18T15:34:23.742435+0000","last_undegraded":"2024-03-18T15:34:23.742435+0000","last_fullsized":"2024-03-18T15:34:23.742435+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:17:15.098291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.743111+0000","last_change":"2024-03-18T15:34:21.755841+0000","last_active":"2024-03-18T15:34:23.743111+0000","last_peered":"2024-03-18T15:34:23.743111+0000","last_clean":"2024-03-18T15:34:23.743111+0000","last_became_active":"2024-03-18T15:34:21.755086+0000","last_became_peered":"2024-03-18T15:34:21.755086+0000","last_unstale":"2024-03-18T15:34:23.743111+0000","last_undegraded":"2024-03-18T15:34:23.743111+0000","last_fullsized":"2024-03-18T15:34:23.743111+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:52:15.262294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.743363+0000","last_change":"2024-03-18T15:34:22.785770+0000","last_active":"2024-03-18T15:34:23.743363+0000","last_peered":"2024-03-18T15:34:23.743363+0000","last_clean":"2024-03-18T15:34:23.743363+0000","last_became_active":"2024-03-18T15:34:22.784901+0000","last_became_peered":"2024-03-18T15:34:22.784901+0000","last_unstale":"2024-03-18T15:34:23.743363+0000","last_undegraded":"2024-03-18T15:34:23.743363+0000","last_fullsized":"2024-03-18T15:34:23.743363+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:33:48.332707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-18T15:34:20.713943+0000","last_change":"2024-03-18T15:34:20.713943+0000","last_active":"2024-03-18T15:34:20.713943+0000","last_peered":"2024-03-18T15:34:20.713943+0000","last_clean":"2024-03-18T15:34:20.713943+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-18T15:34:20.713943+0000","last_undegraded":"2024-03-18T15:34:20.713943+0000","last_fullsized":"2024-03-18T15:34:20.713943+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.743547+0000","last_change":"2024-03-18T15:34:22.778725+0000","last_active":"2024-03-18T15:34:23.743547+0000","last_peered":"2024-03-18T15:34:23.743547+0000","last_clean":"2024-03-18T15:34:23.743547+0000","last_became_active":"2024-03-18T15:34:22.778287+0000","last_became_peered":"2024-03-18T15:34:22.778287+0000","last_unstale":"2024-03-18T15:34:23.743547+0000","last_undegraded":"2024-03-18T15:34:23.743547+0000","last_fullsized":"2024-03-18T15:34:23.743547+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T15:57:43.268841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"29'2","reported_seq":16,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.752602+0000","last_change":"2024-03-18T15:34:21.777078+0000","last_active":"2024-03-18T15:34:23.752602+0000","last_peered":"2024-03-18T15:34:23.752602+0000","last_clean":"2024-03-18T15:34:23.752602+0000","last_became_active":"2024-03-18T15:34:21.776581+0000","last_became_peered":"2024-03-18T15:34:21.776581+0000","last_unstale":"2024-03-18T15:34:23.752602+0000","last_undegraded":"2024-03-18T15:34:23.752602+0000","last_fullsized":"2024-03-18T15:34:23.752602+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:56:42.710235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-18T15:34:21.723958+0000","last_change":"2024-03-18T15:34:21.723958+0000","last_active":"2024-03-18T15:34:21.723958+0000","last_peered":"2024-03-18T15:34:21.723958+0000","last_clean":"2024-03-18T15:34:21.723958+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-18T15:34:21.723958+0000","last_undegraded":"2024-03-18T15:34:21.723958+0000","last_fullsized":"2024-03-18T15:34:21.723958+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"30'1","reported_seq":15,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.750927+0000","last_change":"2024-03-18T15:34:21.769470+0000","last_active":"2024-03-18T15:34:23.750927+0000","last_peered":"2024-03-18T15:34:23.750927+0000","last_clean":"2024-03-18T15:34:23.750927+0000","last_became_active":"2024-03-18T15:34:21.769184+0000","last_became_peered":"2024-03-18T15:34:21.769184+0000","last_unstale":"2024-03-18T15:34:23.750927+0000","last_undegraded":"2024-03-18T15:34:23.750927+0000","last_fullsized":"2024-03-18T15:34:23.750927+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:34:09.363855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.749071+0000","last_change":"2024-03-18T15:34:22.765932+0000","last_active":"2024-03-18T15:34:23.749071+0000","last_peered":"2024-03-18T15:34:23.749071+0000","last_clean":"2024-03-18T15:34:23.749071+0000","last_became_active":"2024-03-18T15:34:22.765701+0000","last_became_peered":"2024-03-18T15:34:22.765701+0000","last_unstale":"2024-03-18T15:34:23.749071+0000","last_undegraded":"2024-03-18T15:34:23.749071+0000","last_fullsized":"2024-03-18T15:34:23.749071+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:02:26.456145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.752247+0000","last_change":"2024-03-18T15:34:21.763930+0000","last_active":"2024-03-18T15:34:23.752247+0000","last_peered":"2024-03-18T15:34:23.752247+0000","last_clean":"2024-03-18T15:34:23.752247+0000","last_became_active":"2024-03-18T15:34:21.763572+0000","last_became_peered":"2024-03-18T15:34:21.763572+0000","last_unstale":"2024-03-18T15:34:23.752247+0000","last_undegraded":"2024-03-18T15:34:23.752247+0000","last_fullsized":"2024-03-18T15:34:23.752247+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:49:56.976995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-18T15:34:21.723958+0000","last_change":"2024-03-18T15:34:21.723958+0000","last_active":"2024-03-18T15:34:21.723958+0000","last_peered":"2024-03-18T15:34:21.723958+0000","last_clean":"2024-03-18T15:34:21.723958+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-18T15:34:21.723958+0000","last_undegraded":"2024-03-18T15:34:21.723958+0000","last_fullsized":"2024-03-18T15:34:21.723958+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.749387+0000","last_change":"2024-03-18T15:34:21.777115+0000","last_active":"2024-03-18T15:34:23.749387+0000","last_peered":"2024-03-18T15:34:23.749387+0000","last_clean":"2024-03-18T15:34:23.749387+0000","last_became_active":"2024-03-18T15:34:21.776733+0000","last_became_peered":"2024-03-18T15:34:21.776733+0000","last_unstale":"2024-03-18T15:34:23.749387+0000","last_undegraded":"2024-03-18T15:34:23.749387+0000","last_fullsized":"2024-03-18T15:34:23.749387+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:01:19.496315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.749248+0000","last_change":"2024-03-18T15:34:22.785592+0000","last_active":"2024-03-18T15:34:23.749248+0000","last_peered":"2024-03-18T15:34:23.749248+0000","last_clean":"2024-03-18T15:34:23.749248+0000","last_became_active":"2024-03-18T15:34:22.785325+0000","last_became_peered":"2024-03-18T15:34:22.785325+0000","last_unstale":"2024-03-18T15:34:23.749248+0000","last_undegraded":"2024-03-18T15:34:23.749248+0000","last_fullsized":"2024-03-18T15:34:23.749248+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:03:09.703931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.751583+0000","last_change":"2024-03-18T15:34:21.769233+0000","last_active":"2024-03-18T15:34:23.751583+0000","last_peered":"2024-03-18T15:34:23.751583+0000","last_clean":"2024-03-18T15:34:23.751583+0000","last_became_active":"2024-03-18T15:34:21.768820+0000","last_became_peered":"2024-03-18T15:34:21.768820+0000","last_unstale":"2024-03-18T15:34:23.751583+0000","last_undegraded":"2024-03-18T15:34:23.751583+0000","last_fullsized":"2024-03-18T15:34:23.751583+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:54:04.203613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-18T15:34:22.781336+0000","last_change":"2024-03-18T15:34:22.768345+0000","last_active":"2024-03-18T15:34:22.781336+0000","last_peered":"2024-03-18T15:34:22.781336+0000","last_clean":"2024-03-18T15:34:22.781336+0000","last_became_active":"2024-03-18T15:34:22.767943+0000","last_became_peered":"2024-03-18T15:34:22.767943+0000","last_unstale":"2024-03-18T15:34:22.781336+0000","last_undegraded":"2024-03-18T15:34:22.781336+0000","last_fullsized":"2024-03-18T15:34:22.781336+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:21:26.063255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.743266+0000","last_change":"2024-03-18T15:34:21.767890+0000","last_active":"2024-03-18T15:34:23.743266+0000","last_peered":"2024-03-18T15:34:23.743266+0000","last_clean":"2024-03-18T15:34:23.743266+0000","last_became_active":"2024-03-18T15:34:21.767243+0000","last_became_peered":"2024-03-18T15:34:21.767243+0000","last_unstale":"2024-03-18T15:34:23.743266+0000","last_undegraded":"2024-03-18T15:34:23.743266+0000","last_fullsized":"2024-03-18T15:34:23.743266+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:10:40.586174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.742865+0000","last_change":"2024-03-18T15:34:22.792199+0000","last_active":"2024-03-18T15:34:23.742865+0000","last_peered":"2024-03-18T15:34:23.742865+0000","last_clean":"2024-03-18T15:34:23.742865+0000","last_became_active":"2024-03-18T15:34:22.791583+0000","last_became_peered":"2024-03-18T15:34:22.791583+0000","last_unstale":"2024-03-18T15:34:23.742865+0000","last_undegraded":"2024-03-18T15:34:23.742865+0000","last_fullsized":"2024-03-18T15:34:23.742865+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:46:07.311590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-18T15:34:22.763729+0000","last_change":"2024-03-18T15:34:21.760360+0000","last_active":"2024-03-18T15:34:22.763729+0000","last_peered":"2024-03-18T15:34:22.763729+0000","last_clean":"2024-03-18T15:34:22.763729+0000","last_became_active":"2024-03-18T15:34:21.759919+0000","last_became_peered":"2024-03-18T15:34:21.759919+0000","last_unstale":"2024-03-18T15:34:22.763729+0000","last_undegraded":"2024-03-18T15:34:22.763729+0000","last_fullsized":"2024-03-18T15:34:22.763729+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:52:33.287817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.743445+0000","last_change":"2024-03-18T15:34:22.764598+0000","last_active":"2024-03-18T15:34:23.743445+0000","last_peered":"2024-03-18T15:34:23.743445+0000","last_clean":"2024-03-18T15:34:23.743445+0000","last_became_active":"2024-03-18T15:34:22.763695+0000","last_became_peered":"2024-03-18T15:34:22.763695+0000","last_unstale":"2024-03-18T15:34:23.743445+0000","last_undegraded":"2024-03-18T15:34:23.743445+0000","last_fullsized":"2024-03-18T15:34:23.743445+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:17:38.633080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-18T15:34:20.713943+0000","last_change":"2024-03-18T15:34:20.713943+0000","last_active":"2024-03-18T15:34:20.713943+0000","last_peered":"2024-03-18T15:34:20.713943+0000","last_clean":"2024-03-18T15:34:20.713943+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-18T15:34:20.713943+0000","last_undegraded":"2024-03-18T15:34:20.713943+0000","last_fullsized":"2024-03-18T15:34:20.713943+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":188,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.745026+0000","last_change":"2024-03-18T15:33:30.766315+0000","last_active":"2024-03-18T15:34:23.745026+0000","last_peered":"2024-03-18T15:34:23.745026+0000","last_clean":"2024-03-18T15:34:23.745026+0000","last_became_active":"2024-03-18T15:33:30.763939+0000","last_became_peered":"2024-03-18T15:33:30.763939+0000","last_unstale":"2024-03-18T15:34:23.745026+0000","last_undegraded":"2024-03-18T15:34:23.745026+0000","last_fullsized":"2024-03-18T15:34:23.745026+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:29.229911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:29.229911+0000","last_clean_scrub_stamp":"2024-03-18T15:33:29.229911+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:10:21.120371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":98,"acting":98,"num_store_stats":6},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":1738,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":54,"acting":54,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509454,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":101374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509454,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27296,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":278528,"data_stored":101374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542159,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":925696,"data_stored":716991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542160,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":335872,"data_stored":126637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542158,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35668,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336172,"statfs":{"total":96636764160,"available":96600240128,"internally_reserved":0,"allocated":331776,"data_stored":121867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542158,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35652,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336188,"statfs":{"total":96636764160,"available":96600256512,"internally_reserved":0,"allocated":319488,"data_stored":120303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542158,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35668,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336172,"statfs":{"total":96636764160,"available":96600240128,"internally_reserved":0,"allocated":331776,"data_stored":121867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542157,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36232,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335608,"statfs":{"total":96636764160,"available":96599662592,"internally_reserved":0,"allocated":909312,"data_stored":710601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-18T15:34:26.742 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-18T15:34:26.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.908+0000 7f3892673640 1 Processor -- start 2024-03-18T15:34:26.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.908+0000 7f3892673640 1 -- start start 2024-03-18T15:34:26.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.908+0000 7f3892673640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f388c0b6c60 0x7f388c0b45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:26.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.908+0000 7f3892673640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f388c148f80 0x7f388c14fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.908+0000 7f3892673640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f388c0b6020 con 0x7f388c072b20 2024-03-18T15:34:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.908+0000 7f3892673640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f388c0b79b0 con 0x7f388c0b6c60 2024-03-18T15:34:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.908+0000 7f3892673640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f388c0b76a0 con 0x7f388c148f80 2024-03-18T15:34:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.908+0000 7f388bfff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f388c0b6c60 0x7f388c0b45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.908+0000 7f388bfff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f388c0b6c60 0x7f388c0b45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45090/0 (socket says 172.21.15.5:45090) 2024-03-18T15:34:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.908+0000 7f388bfff640 1 -- 172.21.15.5:0/4148577246 learned_addr learned my addr 172.21.15.5:0/4148577246 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:26.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.908+0000 7f3890c10640 1 --2- 172.21.15.5:0/4148577246 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f388c148f80 0x7f388c14fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:26.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.908+0000 7f388bfff640 1 -- 172.21.15.5:0/4148577246 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f388c148f80 msgr2=0x7f388c14fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:26.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.908+0000 7f388bfff640 1 --2- 172.21.15.5:0/4148577246 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f388c148f80 0x7f388c14fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:26.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.908+0000 7f388bfff640 1 -- 172.21.15.5:0/4148577246 >> v1:172.21.15.80:6789/0 conn(0x7f388c072b20 legacy=0x7f388c148870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:26.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.908+0000 7f388bfff640 1 -- 172.21.15.5:0/4148577246 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f388c0b7040 con 0x7f388c0b6c60 2024-03-18T15:34:26.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.908+0000 7f3890c10640 1 --2- 172.21.15.5:0/4148577246 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f388c148f80 0x7f388c14fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:26.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.908+0000 7f388bfff640 1 --2- 172.21.15.5:0/4148577246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f388c0b6c60 0x7f388c0b45c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f387c009870 tx=0x7f387c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8feeae5cbe3a1b6b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:26.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.908+0000 7f388affd640 1 -- 172.21.15.5:0/4148577246 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f387c0a7070 con 0x7f388c0b6c60 2024-03-18T15:34:26.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.908+0000 7f388affd640 1 -- 172.21.15.5:0/4148577246 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f387c0b0070 con 0x7f388c0b6c60 2024-03-18T15:34:26.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.908+0000 7f388affd640 1 -- 172.21.15.5:0/4148577246 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f387c0ab890 con 0x7f388c0b6c60 2024-03-18T15:34:26.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.912+0000 7f3892673640 1 -- 172.21.15.5:0/4148577246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f388c0b6c60 msgr2=0x7f388c0b45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:26.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.912+0000 7f3892673640 1 --2- 172.21.15.5:0/4148577246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f388c0b6c60 0x7f388c0b45c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f387c009870 tx=0x7f387c01c490 comp rx=0 tx=0).stop 2024-03-18T15:34:26.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.912+0000 7f3892673640 1 -- 172.21.15.5:0/4148577246 shutdown_connections 2024-03-18T15:34:26.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.912+0000 7f3892673640 1 --2- 172.21.15.5:0/4148577246 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f388c148f80 0x7f388c14fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:26.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.912+0000 7f3892673640 1 --2- 172.21.15.5:0/4148577246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f388c0b6c60 0x7f388c0b45c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:26.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.912+0000 7f3892673640 1 -- 172.21.15.5:0/4148577246 >> 172.21.15.5:0/4148577246 conn(0x7f388c066fc0 msgr2=0x7f388c0673d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:26.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.912+0000 7f3892673640 1 -- 172.21.15.5:0/4148577246 shutdown_connections 2024-03-18T15:34:26.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.912+0000 7f3892673640 1 -- 172.21.15.5:0/4148577246 wait complete. 2024-03-18T15:34:26.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.912+0000 7f3892673640 1 Processor -- start 2024-03-18T15:34:26.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.912+0000 7f3892673640 1 -- start start 2024-03-18T15:34:26.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.912+0000 7f3892673640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f388c072b20 0x7f388c1e18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:26.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.912+0000 7f3892673640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f388c0b6c60 0x7f388c149ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:26.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.912+0000 7f3892673640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f388c142ad0 0x7f388c13cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:26.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.912+0000 7f3892673640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f388c152c90 con 0x7f388c072b20 2024-03-18T15:34:26.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.912+0000 7f3892673640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f388c152b10 con 0x7f388c0b6c60 2024-03-18T15:34:26.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.912+0000 7f3892673640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f388c152e10 con 0x7f388c142ad0 2024-03-18T15:34:26.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.912+0000 7f388bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f388c072b20 0x7f388c1e18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:26.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.912+0000 7f388bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f388c072b20 0x7f388c1e18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45094/0 (socket says 172.21.15.5:45094) 2024-03-18T15:34:26.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.912+0000 7f388bfff640 1 -- 172.21.15.5:0/2798504231 learned_addr learned my addr 172.21.15.5:0/2798504231 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:26.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.912+0000 7f388b7fe640 1 --2- 172.21.15.5:0/2798504231 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f388c0b6c60 0x7f388c149ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:26.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.912+0000 7f3890c10640 1 --2- 172.21.15.5:0/2798504231 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f388c142ad0 0x7f388c13cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:26.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.912+0000 7f388bfff640 1 -- 172.21.15.5:0/2798504231 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f388c142ad0 msgr2=0x7f388c13cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:26.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.912+0000 7f388bfff640 1 --2- 172.21.15.5:0/2798504231 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f388c142ad0 0x7f388c13cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:26.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.912+0000 7f388bfff640 1 -- 172.21.15.5:0/2798504231 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f388c0b6c60 msgr2=0x7f388c149ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:26.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.912+0000 7f388bfff640 1 --2- 172.21.15.5:0/2798504231 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f388c0b6c60 0x7f388c149ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:26.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.912+0000 7f388bfff640 1 -- 172.21.15.5:0/2798504231 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f388c14b490 con 0x7f388c072b20 2024-03-18T15:34:26.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.912+0000 7f3890c10640 1 --2- 172.21.15.5:0/2798504231 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f388c142ad0 0x7f388c13cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:26.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.912+0000 7f388b7fe640 1 --2- 172.21.15.5:0/2798504231 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f388c0b6c60 0x7f388c149ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:26.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.912+0000 7f388bfff640 1 --2- 172.21.15.5:0/2798504231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f388c072b20 0x7f388c1e18c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f387c0ad040 tx=0x7f387c01acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:26.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.912+0000 7f3888ff9640 1 -- 172.21.15.5:0/2798504231 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f387c0b4800 con 0x7f388c072b20 2024-03-18T15:34:26.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.912+0000 7f3888ff9640 1 -- 172.21.15.5:0/2798504231 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f387c0b49c0 con 0x7f388c072b20 2024-03-18T15:34:26.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.912+0000 7f3892673640 1 -- 172.21.15.5:0/2798504231 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f388c14a980 con 0x7f388c072b20 2024-03-18T15:34:26.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.912+0000 7f3888ff9640 1 -- 172.21.15.5:0/2798504231 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f387c0b4db0 con 0x7f388c072b20 2024-03-18T15:34:26.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.912+0000 7f3892673640 1 -- 172.21.15.5:0/2798504231 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f388c14a6a0 con 0x7f388c072b20 2024-03-18T15:34:26.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.916+0000 7f3892673640 1 -- 172.21.15.5:0/2798504231 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f388c04f550 con 0x7f388c072b20 2024-03-18T15:34:26.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.916+0000 7f3888ff9640 1 -- 172.21.15.5:0/2798504231 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f387c0c7440 con 0x7f388c072b20 2024-03-18T15:34:26.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.916+0000 7f3888ff9640 1 --2- 172.21.15.5:0/2798504231 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3878067660 0x7f3878069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:26.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.916+0000 7f3888ff9640 1 -- 172.21.15.5:0/2798504231 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7f387c1201e0 con 0x7f388c072b20 2024-03-18T15:34:26.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.916+0000 7f388b7fe640 1 --2- 172.21.15.5:0/2798504231 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3878067660 0x7f3878069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:26.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.916+0000 7f388b7fe640 1 --2- 172.21.15.5:0/2798504231 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3878067660 0x7f3878069b20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f388c143c90 tx=0x7f3880009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:26.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:26.920+0000 7f3888ff9640 1 -- 172.21.15.5:0/2798504231 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f387c016110 con 0x7f388c072b20 2024-03-18T15:34:27.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.076+0000 7f3892673640 1 -- 172.21.15.5:0/2798504231 --> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f388c14ba90 con 0x7f3878067660 2024-03-18T15:34:27.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.080+0000 7f3888ff9640 1 -- 172.21.15.5:0/2798504231 <== mgr.4110 v2:172.21.15.5:6832/3362034860 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251942 (secure 0 0 0) 0x7f388c14ba90 con 0x7f3878067660 2024-03-18T15:34:27.086 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:34:27.088 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-03-18T15:34:27.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.084+0000 7f3892673640 1 -- 172.21.15.5:0/2798504231 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3878067660 msgr2=0x7f3878069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:27.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.084+0000 7f3892673640 1 --2- 172.21.15.5:0/2798504231 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3878067660 0x7f3878069b20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f388c143c90 tx=0x7f3880009040 comp rx=0 tx=0).stop 2024-03-18T15:34:27.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.084+0000 7f3892673640 1 -- 172.21.15.5:0/2798504231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f388c072b20 msgr2=0x7f388c1e18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:27.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.084+0000 7f3892673640 1 --2- 172.21.15.5:0/2798504231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f388c072b20 0x7f388c1e18c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f387c0ad040 tx=0x7f387c01acd0 comp rx=0 tx=0).stop 2024-03-18T15:34:27.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.084+0000 7f3892673640 1 -- 172.21.15.5:0/2798504231 shutdown_connections 2024-03-18T15:34:27.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.084+0000 7f3892673640 1 --2- 172.21.15.5:0/2798504231 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3878067660 0x7f3878069b20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:27.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.084+0000 7f3892673640 1 --2- 172.21.15.5:0/2798504231 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f388c142ad0 0x7f388c13cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:27.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.084+0000 7f3892673640 1 --2- 172.21.15.5:0/2798504231 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f388c0b6c60 0x7f388c149ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:27.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.084+0000 7f3892673640 1 --2- 172.21.15.5:0/2798504231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f388c072b20 0x7f388c1e18c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:27.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.084+0000 7f3892673640 1 -- 172.21.15.5:0/2798504231 >> 172.21.15.5:0/2798504231 conn(0x7f388c066fc0 msgr2=0x7f388c1449a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:27.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.084+0000 7f3892673640 1 -- 172.21.15.5:0/2798504231 shutdown_connections 2024-03-18T15:34:27.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.084+0000 7f3892673640 1 -- 172.21.15.5:0/2798504231 wait complete. 2024-03-18T15:34:27.103 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":52,"stamp":"2024-03-18T15:34:26.898106+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":123,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":253196,"kb_used_data":3724,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754721524,"statfs":{"total":773094113280,"available":772834840576,"internally_reserved":0,"allocated":3813376,"data_stored":2166320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001359"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:24.259991+0000","last_change":"2024-03-18T15:34:22.788575+0000","last_active":"2024-03-18T15:34:24.259991+0000","last_peered":"2024-03-18T15:34:24.259991+0000","last_clean":"2024-03-18T15:34:24.259991+0000","last_became_active":"2024-03-18T15:34:22.782777+0000","last_became_peered":"2024-03-18T15:34:22.782777+0000","last_unstale":"2024-03-18T15:34:24.259991+0000","last_undegraded":"2024-03-18T15:34:24.259991+0000","last_fullsized":"2024-03-18T15:34:24.259991+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:02:26.456145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-18T15:34:22.783118+0000","last_change":"2024-03-18T15:34:22.774732+0000","last_active":"2024-03-18T15:34:22.783118+0000","last_peered":"2024-03-18T15:34:22.783118+0000","last_clean":"2024-03-18T15:34:22.783118+0000","last_became_active":"2024-03-18T15:34:22.771881+0000","last_became_peered":"2024-03-18T15:34:22.771881+0000","last_unstale":"2024-03-18T15:34:22.783118+0000","last_undegraded":"2024-03-18T15:34:22.783118+0000","last_fullsized":"2024-03-18T15:34:22.783118+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:17:02.398404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:24.173806+0000","last_change":"2024-03-18T15:34:22.778475+0000","last_active":"2024-03-18T15:34:24.173806+0000","last_peered":"2024-03-18T15:34:24.173806+0000","last_clean":"2024-03-18T15:34:24.173806+0000","last_became_active":"2024-03-18T15:34:22.777744+0000","last_became_peered":"2024-03-18T15:34:22.777744+0000","last_unstale":"2024-03-18T15:34:24.173806+0000","last_undegraded":"2024-03-18T15:34:24.173806+0000","last_fullsized":"2024-03-18T15:34:24.173806+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:53:19.453969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.751835+0000","last_change":"2024-03-18T15:34:22.777889+0000","last_active":"2024-03-18T15:34:23.751835+0000","last_peered":"2024-03-18T15:34:23.751835+0000","last_clean":"2024-03-18T15:34:23.751835+0000","last_became_active":"2024-03-18T15:34:22.777438+0000","last_became_peered":"2024-03-18T15:34:22.777438+0000","last_unstale":"2024-03-18T15:34:23.751835+0000","last_undegraded":"2024-03-18T15:34:23.751835+0000","last_fullsized":"2024-03-18T15:34:23.751835+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:16:42.700006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.742750+0000","last_change":"2024-03-18T15:34:22.758037+0000","last_active":"2024-03-18T15:34:23.742750+0000","last_peered":"2024-03-18T15:34:23.742750+0000","last_clean":"2024-03-18T15:34:23.742750+0000","last_became_active":"2024-03-18T15:34:22.757761+0000","last_became_peered":"2024-03-18T15:34:22.757761+0000","last_unstale":"2024-03-18T15:34:23.742750+0000","last_undegraded":"2024-03-18T15:34:23.742750+0000","last_fullsized":"2024-03-18T15:34:23.742750+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:17:47.611508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-18T15:34:22.783560+0000","last_change":"2024-03-18T15:34:22.780952+0000","last_active":"2024-03-18T15:34:22.783560+0000","last_peered":"2024-03-18T15:34:22.783560+0000","last_clean":"2024-03-18T15:34:22.783560+0000","last_became_active":"2024-03-18T15:34:22.780667+0000","last_became_peered":"2024-03-18T15:34:22.780667+0000","last_unstale":"2024-03-18T15:34:22.783560+0000","last_undegraded":"2024-03-18T15:34:22.783560+0000","last_fullsized":"2024-03-18T15:34:22.783560+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:25:38.946261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.744057+0000","last_change":"2024-03-18T15:34:22.786411+0000","last_active":"2024-03-18T15:34:23.744057+0000","last_peered":"2024-03-18T15:34:23.744057+0000","last_clean":"2024-03-18T15:34:23.744057+0000","last_became_active":"2024-03-18T15:34:22.785695+0000","last_became_peered":"2024-03-18T15:34:22.785695+0000","last_unstale":"2024-03-18T15:34:23.744057+0000","last_undegraded":"2024-03-18T15:34:23.744057+0000","last_fullsized":"2024-03-18T15:34:23.744057+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:51:42.881058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.744019+0000","last_change":"2024-03-18T15:34:22.758732+0000","last_active":"2024-03-18T15:34:23.744019+0000","last_peered":"2024-03-18T15:34:23.744019+0000","last_clean":"2024-03-18T15:34:23.744019+0000","last_became_active":"2024-03-18T15:34:22.758033+0000","last_became_peered":"2024-03-18T15:34:22.758033+0000","last_unstale":"2024-03-18T15:34:23.744019+0000","last_undegraded":"2024-03-18T15:34:23.744019+0000","last_fullsized":"2024-03-18T15:34:23.744019+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:49:06.417569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.746246+0000","last_change":"2024-03-18T15:34:22.791072+0000","last_active":"2024-03-18T15:34:23.746246+0000","last_peered":"2024-03-18T15:34:23.746246+0000","last_clean":"2024-03-18T15:34:23.746246+0000","last_became_active":"2024-03-18T15:34:22.790267+0000","last_became_peered":"2024-03-18T15:34:22.790267+0000","last_unstale":"2024-03-18T15:34:23.746246+0000","last_undegraded":"2024-03-18T15:34:23.746246+0000","last_fullsized":"2024-03-18T15:34:23.746246+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:03:05.969919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.751799+0000","last_change":"2024-03-18T15:34:22.773037+0000","last_active":"2024-03-18T15:34:23.751799+0000","last_peered":"2024-03-18T15:34:23.751799+0000","last_clean":"2024-03-18T15:34:23.751799+0000","last_became_active":"2024-03-18T15:34:22.772714+0000","last_became_peered":"2024-03-18T15:34:22.772714+0000","last_unstale":"2024-03-18T15:34:23.751799+0000","last_undegraded":"2024-03-18T15:34:23.751799+0000","last_fullsized":"2024-03-18T15:34:23.751799+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:17:02.398404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.744088+0000","last_change":"2024-03-18T15:34:22.763800+0000","last_active":"2024-03-18T15:34:23.744088+0000","last_peered":"2024-03-18T15:34:23.744088+0000","last_clean":"2024-03-18T15:34:23.744088+0000","last_became_active":"2024-03-18T15:34:22.762781+0000","last_became_peered":"2024-03-18T15:34:22.762781+0000","last_unstale":"2024-03-18T15:34:23.744088+0000","last_undegraded":"2024-03-18T15:34:23.744088+0000","last_fullsized":"2024-03-18T15:34:23.744088+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:36:43.593724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:24.174074+0000","last_change":"2024-03-18T15:34:22.776805+0000","last_active":"2024-03-18T15:34:24.174074+0000","last_peered":"2024-03-18T15:34:24.174074+0000","last_clean":"2024-03-18T15:34:24.174074+0000","last_became_active":"2024-03-18T15:34:22.775273+0000","last_became_peered":"2024-03-18T15:34:22.775273+0000","last_unstale":"2024-03-18T15:34:24.174074+0000","last_undegraded":"2024-03-18T15:34:24.174074+0000","last_fullsized":"2024-03-18T15:34:24.174074+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:51:36.447249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.742897+0000","last_change":"2024-03-18T15:34:22.751395+0000","last_active":"2024-03-18T15:34:23.742897+0000","last_peered":"2024-03-18T15:34:23.742897+0000","last_clean":"2024-03-18T15:34:23.742897+0000","last_became_active":"2024-03-18T15:34:22.751047+0000","last_became_peered":"2024-03-18T15:34:22.751047+0000","last_unstale":"2024-03-18T15:34:23.742897+0000","last_undegraded":"2024-03-18T15:34:23.742897+0000","last_fullsized":"2024-03-18T15:34:23.742897+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:31:27.596543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-18T15:34:22.781158+0000","last_change":"2024-03-18T15:34:22.767530+0000","last_active":"2024-03-18T15:34:22.781158+0000","last_peered":"2024-03-18T15:34:22.781158+0000","last_clean":"2024-03-18T15:34:22.781158+0000","last_became_active":"2024-03-18T15:34:22.762123+0000","last_became_peered":"2024-03-18T15:34:22.762123+0000","last_unstale":"2024-03-18T15:34:22.781158+0000","last_undegraded":"2024-03-18T15:34:22.781158+0000","last_fullsized":"2024-03-18T15:34:22.781158+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:29:09.934918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:24.173219+0000","last_change":"2024-03-18T15:34:22.797770+0000","last_active":"2024-03-18T15:34:24.173219+0000","last_peered":"2024-03-18T15:34:24.173219+0000","last_clean":"2024-03-18T15:34:24.173219+0000","last_became_active":"2024-03-18T15:34:22.797112+0000","last_became_peered":"2024-03-18T15:34:22.797112+0000","last_unstale":"2024-03-18T15:34:24.173219+0000","last_undegraded":"2024-03-18T15:34:24.173219+0000","last_fullsized":"2024-03-18T15:34:24.173219+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T15:35:44.715099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.745086+0000","last_change":"2024-03-18T15:34:22.768015+0000","last_active":"2024-03-18T15:34:23.745086+0000","last_peered":"2024-03-18T15:34:23.745086+0000","last_clean":"2024-03-18T15:34:23.745086+0000","last_became_active":"2024-03-18T15:34:22.767310+0000","last_became_peered":"2024-03-18T15:34:22.767310+0000","last_unstale":"2024-03-18T15:34:23.745086+0000","last_undegraded":"2024-03-18T15:34:23.745086+0000","last_fullsized":"2024-03-18T15:34:23.745086+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:18:33.478792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.744439+0000","last_change":"2024-03-18T15:34:22.790197+0000","last_active":"2024-03-18T15:34:23.744439+0000","last_peered":"2024-03-18T15:34:23.744439+0000","last_clean":"2024-03-18T15:34:23.744439+0000","last_became_active":"2024-03-18T15:34:22.789575+0000","last_became_peered":"2024-03-18T15:34:22.789575+0000","last_unstale":"2024-03-18T15:34:23.744439+0000","last_undegraded":"2024-03-18T15:34:23.744439+0000","last_fullsized":"2024-03-18T15:34:23.744439+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:25:26.289800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-18T15:34:22.783145+0000","last_change":"2024-03-18T15:34:22.775054+0000","last_active":"2024-03-18T15:34:22.783145+0000","last_peered":"2024-03-18T15:34:22.783145+0000","last_clean":"2024-03-18T15:34:22.783145+0000","last_became_active":"2024-03-18T15:34:22.772194+0000","last_became_peered":"2024-03-18T15:34:22.772194+0000","last_unstale":"2024-03-18T15:34:22.783145+0000","last_undegraded":"2024-03-18T15:34:22.783145+0000","last_fullsized":"2024-03-18T15:34:22.783145+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:16:42.700006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-18T15:34:22.783079+0000","last_change":"2024-03-18T15:34:22.772172+0000","last_active":"2024-03-18T15:34:22.783079+0000","last_peered":"2024-03-18T15:34:22.783079+0000","last_clean":"2024-03-18T15:34:22.783079+0000","last_became_active":"2024-03-18T15:34:22.771755+0000","last_became_peered":"2024-03-18T15:34:22.771755+0000","last_unstale":"2024-03-18T15:34:22.783079+0000","last_undegraded":"2024-03-18T15:34:22.783079+0000","last_fullsized":"2024-03-18T15:34:22.783079+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:57:00.519939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.743530+0000","last_change":"2024-03-18T15:34:22.780542+0000","last_active":"2024-03-18T15:34:23.743530+0000","last_peered":"2024-03-18T15:34:23.743530+0000","last_clean":"2024-03-18T15:34:23.743530+0000","last_became_active":"2024-03-18T15:34:22.780261+0000","last_became_peered":"2024-03-18T15:34:22.780261+0000","last_unstale":"2024-03-18T15:34:23.743530+0000","last_undegraded":"2024-03-18T15:34:23.743530+0000","last_fullsized":"2024-03-18T15:34:23.743530+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:24:52.230317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.744707+0000","last_change":"2024-03-18T15:34:22.772477+0000","last_active":"2024-03-18T15:34:23.744707+0000","last_peered":"2024-03-18T15:34:23.744707+0000","last_clean":"2024-03-18T15:34:23.744707+0000","last_became_active":"2024-03-18T15:34:22.771735+0000","last_became_peered":"2024-03-18T15:34:22.771735+0000","last_unstale":"2024-03-18T15:34:23.744707+0000","last_undegraded":"2024-03-18T15:34:23.744707+0000","last_fullsized":"2024-03-18T15:34:23.744707+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:52:34.297832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.744641+0000","last_change":"2024-03-18T15:34:22.763214+0000","last_active":"2024-03-18T15:34:23.744641+0000","last_peered":"2024-03-18T15:34:23.744641+0000","last_clean":"2024-03-18T15:34:23.744641+0000","last_became_active":"2024-03-18T15:34:22.762356+0000","last_became_peered":"2024-03-18T15:34:22.762356+0000","last_unstale":"2024-03-18T15:34:23.744641+0000","last_undegraded":"2024-03-18T15:34:23.744641+0000","last_fullsized":"2024-03-18T15:34:23.744641+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:43:37.362381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.749211+0000","last_change":"2024-03-18T15:34:22.770613+0000","last_active":"2024-03-18T15:34:23.749211+0000","last_peered":"2024-03-18T15:34:23.749211+0000","last_clean":"2024-03-18T15:34:23.749211+0000","last_became_active":"2024-03-18T15:34:22.770340+0000","last_became_peered":"2024-03-18T15:34:22.770340+0000","last_unstale":"2024-03-18T15:34:23.749211+0000","last_undegraded":"2024-03-18T15:34:23.749211+0000","last_fullsized":"2024-03-18T15:34:23.749211+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:01:17.226430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:24.173546+0000","last_change":"2024-03-18T15:34:22.764681+0000","last_active":"2024-03-18T15:34:24.173546+0000","last_peered":"2024-03-18T15:34:24.173546+0000","last_clean":"2024-03-18T15:34:24.173546+0000","last_became_active":"2024-03-18T15:34:22.763966+0000","last_became_peered":"2024-03-18T15:34:22.763966+0000","last_unstale":"2024-03-18T15:34:24.173546+0000","last_undegraded":"2024-03-18T15:34:24.173546+0000","last_fullsized":"2024-03-18T15:34:24.173546+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:04:00.068325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.752145+0000","last_change":"2024-03-18T15:34:22.766888+0000","last_active":"2024-03-18T15:34:23.752145+0000","last_peered":"2024-03-18T15:34:23.752145+0000","last_clean":"2024-03-18T15:34:23.752145+0000","last_became_active":"2024-03-18T15:34:22.766610+0000","last_became_peered":"2024-03-18T15:34:22.766610+0000","last_unstale":"2024-03-18T15:34:23.752145+0000","last_undegraded":"2024-03-18T15:34:23.752145+0000","last_fullsized":"2024-03-18T15:34:23.752145+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:02:32.023342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-18T15:34:22.783440+0000","last_change":"2024-03-18T15:34:22.776017+0000","last_active":"2024-03-18T15:34:22.783440+0000","last_peered":"2024-03-18T15:34:22.783440+0000","last_clean":"2024-03-18T15:34:22.783440+0000","last_became_active":"2024-03-18T15:34:22.771540+0000","last_became_peered":"2024-03-18T15:34:22.771540+0000","last_unstale":"2024-03-18T15:34:22.783440+0000","last_undegraded":"2024-03-18T15:34:22.783440+0000","last_fullsized":"2024-03-18T15:34:22.783440+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:58:39.514486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.752850+0000","last_change":"2024-03-18T15:34:22.779619+0000","last_active":"2024-03-18T15:34:23.752850+0000","last_peered":"2024-03-18T15:34:23.752850+0000","last_clean":"2024-03-18T15:34:23.752850+0000","last_became_active":"2024-03-18T15:34:22.779331+0000","last_became_peered":"2024-03-18T15:34:22.779331+0000","last_unstale":"2024-03-18T15:34:23.752850+0000","last_undegraded":"2024-03-18T15:34:23.752850+0000","last_fullsized":"2024-03-18T15:34:23.752850+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:25:38.946261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:24.173921+0000","last_change":"2024-03-18T15:34:22.779584+0000","last_active":"2024-03-18T15:34:24.173921+0000","last_peered":"2024-03-18T15:34:24.173921+0000","last_clean":"2024-03-18T15:34:24.173921+0000","last_became_active":"2024-03-18T15:34:22.778407+0000","last_became_peered":"2024-03-18T15:34:22.778407+0000","last_unstale":"2024-03-18T15:34:24.173921+0000","last_undegraded":"2024-03-18T15:34:24.173921+0000","last_fullsized":"2024-03-18T15:34:24.173921+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:10:12.933349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.745049+0000","last_change":"2024-03-18T15:34:22.766061+0000","last_active":"2024-03-18T15:34:23.745049+0000","last_peered":"2024-03-18T15:34:23.745049+0000","last_clean":"2024-03-18T15:34:23.745049+0000","last_became_active":"2024-03-18T15:34:22.764782+0000","last_became_peered":"2024-03-18T15:34:22.764782+0000","last_unstale":"2024-03-18T15:34:23.745049+0000","last_undegraded":"2024-03-18T15:34:23.745049+0000","last_fullsized":"2024-03-18T15:34:23.745049+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:29:09.934918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.744517+0000","last_change":"2024-03-18T15:34:22.785047+0000","last_active":"2024-03-18T15:34:23.744517+0000","last_peered":"2024-03-18T15:34:23.744517+0000","last_clean":"2024-03-18T15:34:23.744517+0000","last_became_active":"2024-03-18T15:34:22.784398+0000","last_became_peered":"2024-03-18T15:34:22.784398+0000","last_unstale":"2024-03-18T15:34:23.744517+0000","last_undegraded":"2024-03-18T15:34:23.744517+0000","last_fullsized":"2024-03-18T15:34:23.744517+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:20:45.678969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:24.259849+0000","last_change":"2024-03-18T15:34:22.784170+0000","last_active":"2024-03-18T15:34:24.259849+0000","last_peered":"2024-03-18T15:34:24.259849+0000","last_clean":"2024-03-18T15:34:24.259849+0000","last_became_active":"2024-03-18T15:34:22.783400+0000","last_became_peered":"2024-03-18T15:34:22.783400+0000","last_unstale":"2024-03-18T15:34:24.259849+0000","last_undegraded":"2024-03-18T15:34:24.259849+0000","last_fullsized":"2024-03-18T15:34:24.259849+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:06:03.222687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.744525+0000","last_change":"2024-03-18T15:34:22.776424+0000","last_active":"2024-03-18T15:34:23.744525+0000","last_peered":"2024-03-18T15:34:23.744525+0000","last_clean":"2024-03-18T15:34:23.744525+0000","last_became_active":"2024-03-18T15:34:22.775691+0000","last_became_peered":"2024-03-18T15:34:22.775691+0000","last_unstale":"2024-03-18T15:34:23.744525+0000","last_undegraded":"2024-03-18T15:34:23.744525+0000","last_fullsized":"2024-03-18T15:34:23.744525+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:57:53.266016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.749412+0000","last_change":"2024-03-18T15:34:22.777517+0000","last_active":"2024-03-18T15:34:23.749412+0000","last_peered":"2024-03-18T15:34:23.749412+0000","last_clean":"2024-03-18T15:34:23.749412+0000","last_became_active":"2024-03-18T15:34:22.777084+0000","last_became_peered":"2024-03-18T15:34:22.777084+0000","last_unstale":"2024-03-18T15:34:23.749412+0000","last_undegraded":"2024-03-18T15:34:23.749412+0000","last_fullsized":"2024-03-18T15:34:23.749412+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:19:13.472867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"29'4","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:24.260260+0000","last_change":"2024-03-18T15:34:21.775490+0000","last_active":"2024-03-18T15:34:24.260260+0000","last_peered":"2024-03-18T15:34:24.260260+0000","last_clean":"2024-03-18T15:34:24.260260+0000","last_became_active":"2024-03-18T15:34:21.774778+0000","last_became_peered":"2024-03-18T15:34:21.774778+0000","last_unstale":"2024-03-18T15:34:24.260260+0000","last_undegraded":"2024-03-18T15:34:24.260260+0000","last_fullsized":"2024-03-18T15:34:24.260260+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:03:37.411256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:24.260022+0000","last_change":"2024-03-18T15:34:22.792901+0000","last_active":"2024-03-18T15:34:24.260022+0000","last_peered":"2024-03-18T15:34:24.260022+0000","last_clean":"2024-03-18T15:34:24.260022+0000","last_became_active":"2024-03-18T15:34:22.792244+0000","last_became_peered":"2024-03-18T15:34:22.792244+0000","last_unstale":"2024-03-18T15:34:24.260022+0000","last_undegraded":"2024-03-18T15:34:24.260022+0000","last_fullsized":"2024-03-18T15:34:24.260022+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:01:17.226430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"29'1","reported_seq":15,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.749280+0000","last_change":"2024-03-18T15:34:21.771464+0000","last_active":"2024-03-18T15:34:23.749280+0000","last_peered":"2024-03-18T15:34:23.749280+0000","last_clean":"2024-03-18T15:34:23.749280+0000","last_became_active":"2024-03-18T15:34:21.771234+0000","last_became_peered":"2024-03-18T15:34:21.771234+0000","last_unstale":"2024-03-18T15:34:23.749280+0000","last_undegraded":"2024-03-18T15:34:23.749280+0000","last_fullsized":"2024-03-18T15:34:23.749280+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T15:57:42.258826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.745934+0000","last_change":"2024-03-18T15:34:22.785781+0000","last_active":"2024-03-18T15:34:23.745934+0000","last_peered":"2024-03-18T15:34:23.745934+0000","last_clean":"2024-03-18T15:34:23.745934+0000","last_became_active":"2024-03-18T15:34:22.785119+0000","last_became_peered":"2024-03-18T15:34:22.785119+0000","last_unstale":"2024-03-18T15:34:23.745934+0000","last_undegraded":"2024-03-18T15:34:23.745934+0000","last_fullsized":"2024-03-18T15:34:23.745934+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:57:00.519939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"29'2","reported_seq":14,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-18T15:34:22.775760+0000","last_change":"2024-03-18T15:34:21.770272+0000","last_active":"2024-03-18T15:34:22.775760+0000","last_peered":"2024-03-18T15:34:22.775760+0000","last_clean":"2024-03-18T15:34:22.775760+0000","last_became_active":"2024-03-18T15:34:21.769876+0000","last_became_peered":"2024-03-18T15:34:21.769876+0000","last_unstale":"2024-03-18T15:34:22.775760+0000","last_undegraded":"2024-03-18T15:34:22.775760+0000","last_fullsized":"2024-03-18T15:34:22.775760+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:09:09.582374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.752782+0000","last_change":"2024-03-18T15:34:22.779629+0000","last_active":"2024-03-18T15:34:23.752782+0000","last_peered":"2024-03-18T15:34:23.752782+0000","last_clean":"2024-03-18T15:34:23.752782+0000","last_became_active":"2024-03-18T15:34:22.778145+0000","last_became_peered":"2024-03-18T15:34:22.778145+0000","last_unstale":"2024-03-18T15:34:23.752782+0000","last_undegraded":"2024-03-18T15:34:23.752782+0000","last_fullsized":"2024-03-18T15:34:23.752782+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:13:16.770552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"29'2","reported_seq":16,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.743679+0000","last_change":"2024-03-18T15:34:21.765588+0000","last_active":"2024-03-18T15:34:23.743679+0000","last_peered":"2024-03-18T15:34:23.743679+0000","last_clean":"2024-03-18T15:34:23.743679+0000","last_became_active":"2024-03-18T15:34:21.764531+0000","last_became_peered":"2024-03-18T15:34:21.764531+0000","last_unstale":"2024-03-18T15:34:23.743679+0000","last_undegraded":"2024-03-18T15:34:23.743679+0000","last_fullsized":"2024-03-18T15:34:23.743679+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:59:41.899954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.744918+0000","last_change":"2024-03-18T15:34:22.792570+0000","last_active":"2024-03-18T15:34:23.744918+0000","last_peered":"2024-03-18T15:34:23.744918+0000","last_clean":"2024-03-18T15:34:23.744918+0000","last_became_active":"2024-03-18T15:34:22.792028+0000","last_became_peered":"2024-03-18T15:34:22.792028+0000","last_unstale":"2024-03-18T15:34:23.744918+0000","last_undegraded":"2024-03-18T15:34:23.744918+0000","last_fullsized":"2024-03-18T15:34:23.744918+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:39:27.621959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"29'1","reported_seq":15,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.745361+0000","last_change":"2024-03-18T15:34:21.754670+0000","last_active":"2024-03-18T15:34:23.745361+0000","last_peered":"2024-03-18T15:34:23.745361+0000","last_clean":"2024-03-18T15:34:23.745361+0000","last_became_active":"2024-03-18T15:34:21.753341+0000","last_became_peered":"2024-03-18T15:34:21.753341+0000","last_unstale":"2024-03-18T15:34:23.745361+0000","last_undegraded":"2024-03-18T15:34:23.745361+0000","last_fullsized":"2024-03-18T15:34:23.745361+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:03:23.511853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:24.173321+0000","last_change":"2024-03-18T15:34:22.770852+0000","last_active":"2024-03-18T15:34:24.173321+0000","last_peered":"2024-03-18T15:34:24.173321+0000","last_clean":"2024-03-18T15:34:24.173321+0000","last_became_active":"2024-03-18T15:34:22.770085+0000","last_became_peered":"2024-03-18T15:34:22.770085+0000","last_unstale":"2024-03-18T15:34:24.173321+0000","last_undegraded":"2024-03-18T15:34:24.173321+0000","last_fullsized":"2024-03-18T15:34:24.173321+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:09:53.566419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"29'1","reported_seq":15,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.744929+0000","last_change":"2024-03-18T15:34:21.754688+0000","last_active":"2024-03-18T15:34:23.744929+0000","last_peered":"2024-03-18T15:34:23.744929+0000","last_clean":"2024-03-18T15:34:23.744929+0000","last_became_active":"2024-03-18T15:34:21.753425+0000","last_became_peered":"2024-03-18T15:34:21.753425+0000","last_unstale":"2024-03-18T15:34:23.744929+0000","last_undegraded":"2024-03-18T15:34:23.744929+0000","last_fullsized":"2024-03-18T15:34:23.744929+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:53:25.674607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.742773+0000","last_change":"2024-03-18T15:34:22.770699+0000","last_active":"2024-03-18T15:34:23.742773+0000","last_peered":"2024-03-18T15:34:23.742773+0000","last_clean":"2024-03-18T15:34:23.742773+0000","last_became_active":"2024-03-18T15:34:22.769967+0000","last_became_peered":"2024-03-18T15:34:22.769967+0000","last_unstale":"2024-03-18T15:34:23.742773+0000","last_undegraded":"2024-03-18T15:34:23.742773+0000","last_fullsized":"2024-03-18T15:34:23.742773+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:22:56.512250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-18T15:34:22.756784+0000","last_change":"2024-03-18T15:34:21.761313+0000","last_active":"2024-03-18T15:34:22.756784+0000","last_peered":"2024-03-18T15:34:22.756784+0000","last_clean":"2024-03-18T15:34:22.756784+0000","last_became_active":"2024-03-18T15:34:21.760662+0000","last_became_peered":"2024-03-18T15:34:21.760662+0000","last_unstale":"2024-03-18T15:34:22.756784+0000","last_undegraded":"2024-03-18T15:34:22.756784+0000","last_fullsized":"2024-03-18T15:34:22.756784+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:44:05.246702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.751740+0000","last_change":"2024-03-18T15:34:22.768121+0000","last_active":"2024-03-18T15:34:23.751740+0000","last_peered":"2024-03-18T15:34:23.751740+0000","last_clean":"2024-03-18T15:34:23.751740+0000","last_became_active":"2024-03-18T15:34:22.767794+0000","last_became_peered":"2024-03-18T15:34:22.767794+0000","last_unstale":"2024-03-18T15:34:23.751740+0000","last_undegraded":"2024-03-18T15:34:23.751740+0000","last_fullsized":"2024-03-18T15:34:23.751740+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:03:05.969919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.742642+0000","last_change":"2024-03-18T15:34:21.769291+0000","last_active":"2024-03-18T15:34:23.742642+0000","last_peered":"2024-03-18T15:34:23.742642+0000","last_clean":"2024-03-18T15:34:23.742642+0000","last_became_active":"2024-03-18T15:34:21.768605+0000","last_became_peered":"2024-03-18T15:34:21.768605+0000","last_unstale":"2024-03-18T15:34:23.742642+0000","last_undegraded":"2024-03-18T15:34:23.742642+0000","last_fullsized":"2024-03-18T15:34:23.742642+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:05:37.023288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:24.174108+0000","last_change":"2024-03-18T15:34:22.773049+0000","last_active":"2024-03-18T15:34:24.174108+0000","last_peered":"2024-03-18T15:34:24.174108+0000","last_clean":"2024-03-18T15:34:24.174108+0000","last_became_active":"2024-03-18T15:34:22.772311+0000","last_became_peered":"2024-03-18T15:34:22.772311+0000","last_unstale":"2024-03-18T15:34:24.174108+0000","last_undegraded":"2024-03-18T15:34:24.174108+0000","last_fullsized":"2024-03-18T15:34:24.174108+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:33:34.478232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-18T15:34:22.756998+0000","last_change":"2024-03-18T15:34:21.760191+0000","last_active":"2024-03-18T15:34:22.756998+0000","last_peered":"2024-03-18T15:34:22.756998+0000","last_clean":"2024-03-18T15:34:22.756998+0000","last_became_active":"2024-03-18T15:34:21.759764+0000","last_became_peered":"2024-03-18T15:34:21.759764+0000","last_unstale":"2024-03-18T15:34:22.756998+0000","last_undegraded":"2024-03-18T15:34:22.756998+0000","last_fullsized":"2024-03-18T15:34:22.756998+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:46:06.301575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-18T15:34:22.783124+0000","last_change":"2024-03-18T15:34:22.772207+0000","last_active":"2024-03-18T15:34:22.783124+0000","last_peered":"2024-03-18T15:34:22.783124+0000","last_clean":"2024-03-18T15:34:22.783124+0000","last_became_active":"2024-03-18T15:34:22.771830+0000","last_became_peered":"2024-03-18T15:34:22.771830+0000","last_unstale":"2024-03-18T15:34:22.783124+0000","last_undegraded":"2024-03-18T15:34:22.783124+0000","last_fullsized":"2024-03-18T15:34:22.783124+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:03:05.969919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.749382+0000","last_change":"2024-03-18T15:34:21.758575+0000","last_active":"2024-03-18T15:34:23.749382+0000","last_peered":"2024-03-18T15:34:23.749382+0000","last_clean":"2024-03-18T15:34:23.749382+0000","last_became_active":"2024-03-18T15:34:21.758127+0000","last_became_peered":"2024-03-18T15:34:21.758127+0000","last_unstale":"2024-03-18T15:34:23.749382+0000","last_undegraded":"2024-03-18T15:34:23.749382+0000","last_fullsized":"2024-03-18T15:34:23.749382+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:03:37.411256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-18T15:34:22.783527+0000","last_change":"2024-03-18T15:34:22.776364+0000","last_active":"2024-03-18T15:34:22.783527+0000","last_peered":"2024-03-18T15:34:22.783527+0000","last_clean":"2024-03-18T15:34:22.783527+0000","last_became_active":"2024-03-18T15:34:22.775298+0000","last_became_peered":"2024-03-18T15:34:22.775298+0000","last_unstale":"2024-03-18T15:34:22.783527+0000","last_undegraded":"2024-03-18T15:34:22.783527+0000","last_fullsized":"2024-03-18T15:34:22.783527+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:13:16.770552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"29'3","reported_seq":17,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:24.174105+0000","last_change":"2024-03-18T15:34:21.772763+0000","last_active":"2024-03-18T15:34:24.174105+0000","last_peered":"2024-03-18T15:34:24.174105+0000","last_clean":"2024-03-18T15:34:24.174105+0000","last_became_active":"2024-03-18T15:34:21.772015+0000","last_became_peered":"2024-03-18T15:34:21.772015+0000","last_unstale":"2024-03-18T15:34:24.174105+0000","last_undegraded":"2024-03-18T15:34:24.174105+0000","last_fullsized":"2024-03-18T15:34:24.174105+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:01:16.216414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:24.173727+0000","last_change":"2024-03-18T15:34:22.777255+0000","last_active":"2024-03-18T15:34:24.173727+0000","last_peered":"2024-03-18T15:34:24.173727+0000","last_clean":"2024-03-18T15:34:24.173727+0000","last_became_active":"2024-03-18T15:34:22.776450+0000","last_became_peered":"2024-03-18T15:34:22.776450+0000","last_unstale":"2024-03-18T15:34:24.173727+0000","last_undegraded":"2024-03-18T15:34:24.173727+0000","last_fullsized":"2024-03-18T15:34:24.173727+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:26:47.856610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"29'6","reported_seq":18,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-18T15:34:22.794071+0000","last_change":"2024-03-18T15:34:21.765099+0000","last_active":"2024-03-18T15:34:22.794071+0000","last_peered":"2024-03-18T15:34:22.794071+0000","last_clean":"2024-03-18T15:34:22.794071+0000","last_became_active":"2024-03-18T15:34:21.764694+0000","last_became_peered":"2024-03-18T15:34:21.764694+0000","last_unstale":"2024-03-18T15:34:22.794071+0000","last_undegraded":"2024-03-18T15:34:22.794071+0000","last_fullsized":"2024-03-18T15:34:22.794071+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:22:55.502235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.751476+0000","last_change":"2024-03-18T15:34:22.779939+0000","last_active":"2024-03-18T15:34:23.751476+0000","last_peered":"2024-03-18T15:34:23.751476+0000","last_clean":"2024-03-18T15:34:23.751476+0000","last_became_active":"2024-03-18T15:34:22.778840+0000","last_became_peered":"2024-03-18T15:34:22.778840+0000","last_unstale":"2024-03-18T15:34:23.751476+0000","last_undegraded":"2024-03-18T15:34:23.751476+0000","last_fullsized":"2024-03-18T15:34:23.751476+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:57:00.519939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"29'2","reported_seq":16,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.745449+0000","last_change":"2024-03-18T15:34:21.769150+0000","last_active":"2024-03-18T15:34:23.745449+0000","last_peered":"2024-03-18T15:34:23.745449+0000","last_clean":"2024-03-18T15:34:23.745449+0000","last_became_active":"2024-03-18T15:34:21.768408+0000","last_became_peered":"2024-03-18T15:34:21.768408+0000","last_unstale":"2024-03-18T15:34:23.745449+0000","last_undegraded":"2024-03-18T15:34:23.745449+0000","last_fullsized":"2024-03-18T15:34:23.745449+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:55:30.386254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-18T15:34:22.783216+0000","last_change":"2024-03-18T15:34:22.776102+0000","last_active":"2024-03-18T15:34:22.783216+0000","last_peered":"2024-03-18T15:34:22.783216+0000","last_clean":"2024-03-18T15:34:22.783216+0000","last_became_active":"2024-03-18T15:34:22.768547+0000","last_became_peered":"2024-03-18T15:34:22.768547+0000","last_unstale":"2024-03-18T15:34:22.783216+0000","last_undegraded":"2024-03-18T15:34:22.783216+0000","last_fullsized":"2024-03-18T15:34:22.783216+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:02:32.023342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.751528+0000","last_change":"2024-03-18T15:34:21.762710+0000","last_active":"2024-03-18T15:34:23.751528+0000","last_peered":"2024-03-18T15:34:23.751528+0000","last_clean":"2024-03-18T15:34:23.751528+0000","last_became_active":"2024-03-18T15:34:21.762295+0000","last_became_peered":"2024-03-18T15:34:21.762295+0000","last_unstale":"2024-03-18T15:34:23.751528+0000","last_undegraded":"2024-03-18T15:34:23.751528+0000","last_fullsized":"2024-03-18T15:34:23.751528+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:55:00.926699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.744897+0000","last_change":"2024-03-18T15:34:22.792550+0000","last_active":"2024-03-18T15:34:23.744897+0000","last_peered":"2024-03-18T15:34:23.744897+0000","last_clean":"2024-03-18T15:34:23.744897+0000","last_became_active":"2024-03-18T15:34:22.791889+0000","last_became_peered":"2024-03-18T15:34:22.791889+0000","last_unstale":"2024-03-18T15:34:23.744897+0000","last_undegraded":"2024-03-18T15:34:23.744897+0000","last_fullsized":"2024-03-18T15:34:23.744897+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:11:46.630503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-18T15:34:22.757106+0000","last_change":"2024-03-18T15:34:21.771889+0000","last_active":"2024-03-18T15:34:22.757106+0000","last_peered":"2024-03-18T15:34:22.757106+0000","last_clean":"2024-03-18T15:34:22.757106+0000","last_became_active":"2024-03-18T15:34:21.771441+0000","last_became_peered":"2024-03-18T15:34:21.771441+0000","last_unstale":"2024-03-18T15:34:22.757106+0000","last_undegraded":"2024-03-18T15:34:22.757106+0000","last_fullsized":"2024-03-18T15:34:22.757106+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:43:36.352366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:24.259927+0000","last_change":"2024-03-18T15:34:22.797037+0000","last_active":"2024-03-18T15:34:24.259927+0000","last_peered":"2024-03-18T15:34:24.259927+0000","last_clean":"2024-03-18T15:34:24.259927+0000","last_became_active":"2024-03-18T15:34:22.796381+0000","last_became_peered":"2024-03-18T15:34:22.796381+0000","last_unstale":"2024-03-18T15:34:24.259927+0000","last_undegraded":"2024-03-18T15:34:24.259927+0000","last_fullsized":"2024-03-18T15:34:24.259927+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:12:13.876016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"29'8","reported_seq":20,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-18T15:34:22.783177+0000","last_change":"2024-03-18T15:34:21.764809+0000","last_active":"2024-03-18T15:34:22.783177+0000","last_peered":"2024-03-18T15:34:22.783177+0000","last_clean":"2024-03-18T15:34:22.783177+0000","last_became_active":"2024-03-18T15:34:21.764537+0000","last_became_peered":"2024-03-18T15:34:21.764537+0000","last_unstale":"2024-03-18T15:34:22.783177+0000","last_undegraded":"2024-03-18T15:34:22.783177+0000","last_fullsized":"2024-03-18T15:34:22.783177+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:49:56.976995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.745893+0000","last_change":"2024-03-18T15:34:22.777789+0000","last_active":"2024-03-18T15:34:23.745893+0000","last_peered":"2024-03-18T15:34:23.745893+0000","last_clean":"2024-03-18T15:34:23.745893+0000","last_became_active":"2024-03-18T15:34:22.777052+0000","last_became_peered":"2024-03-18T15:34:22.777052+0000","last_unstale":"2024-03-18T15:34:23.745893+0000","last_undegraded":"2024-03-18T15:34:23.745893+0000","last_fullsized":"2024-03-18T15:34:23.745893+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:21:26.063255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"29'1","reported_seq":15,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.752480+0000","last_change":"2024-03-18T15:34:21.771611+0000","last_active":"2024-03-18T15:34:23.752480+0000","last_peered":"2024-03-18T15:34:23.752480+0000","last_clean":"2024-03-18T15:34:23.752480+0000","last_became_active":"2024-03-18T15:34:21.769590+0000","last_became_peered":"2024-03-18T15:34:21.769590+0000","last_unstale":"2024-03-18T15:34:23.752480+0000","last_undegraded":"2024-03-18T15:34:23.752480+0000","last_fullsized":"2024-03-18T15:34:23.752480+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:34:03.948098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:24.173831+0000","last_change":"2024-03-18T15:34:22.793408+0000","last_active":"2024-03-18T15:34:24.173831+0000","last_peered":"2024-03-18T15:34:24.173831+0000","last_clean":"2024-03-18T15:34:24.173831+0000","last_became_active":"2024-03-18T15:34:22.792872+0000","last_became_peered":"2024-03-18T15:34:22.792872+0000","last_unstale":"2024-03-18T15:34:24.173831+0000","last_undegraded":"2024-03-18T15:34:24.173831+0000","last_fullsized":"2024-03-18T15:34:24.173831+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:01:05.940909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"29'2","reported_seq":16,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.745821+0000","last_change":"2024-03-18T15:34:21.769268+0000","last_active":"2024-03-18T15:34:23.745821+0000","last_peered":"2024-03-18T15:34:23.745821+0000","last_clean":"2024-03-18T15:34:23.745821+0000","last_became_active":"2024-03-18T15:34:21.768518+0000","last_became_peered":"2024-03-18T15:34:21.768518+0000","last_unstale":"2024-03-18T15:34:23.745821+0000","last_undegraded":"2024-03-18T15:34:23.745821+0000","last_fullsized":"2024-03-18T15:34:23.745821+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:53:02.656952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.752032+0000","last_change":"2024-03-18T15:34:22.777582+0000","last_active":"2024-03-18T15:34:23.752032+0000","last_peered":"2024-03-18T15:34:23.752032+0000","last_clean":"2024-03-18T15:34:23.752032+0000","last_became_active":"2024-03-18T15:34:22.777227+0000","last_became_peered":"2024-03-18T15:34:22.777227+0000","last_unstale":"2024-03-18T15:34:23.752032+0000","last_undegraded":"2024-03-18T15:34:23.752032+0000","last_fullsized":"2024-03-18T15:34:23.752032+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:58:39.514486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"29'1","reported_seq":15,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.746021+0000","last_change":"2024-03-18T15:34:21.758424+0000","last_active":"2024-03-18T15:34:23.746021+0000","last_peered":"2024-03-18T15:34:23.746021+0000","last_clean":"2024-03-18T15:34:23.746021+0000","last_became_active":"2024-03-18T15:34:21.756927+0000","last_became_peered":"2024-03-18T15:34:21.756927+0000","last_unstale":"2024-03-18T15:34:23.746021+0000","last_undegraded":"2024-03-18T15:34:23.746021+0000","last_fullsized":"2024-03-18T15:34:23.746021+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:20:41.054645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.743489+0000","last_change":"2024-03-18T15:34:22.764733+0000","last_active":"2024-03-18T15:34:23.743489+0000","last_peered":"2024-03-18T15:34:23.743489+0000","last_clean":"2024-03-18T15:34:23.743489+0000","last_became_active":"2024-03-18T15:34:22.764348+0000","last_became_peered":"2024-03-18T15:34:22.764348+0000","last_unstale":"2024-03-18T15:34:23.743489+0000","last_undegraded":"2024-03-18T15:34:23.743489+0000","last_fullsized":"2024-03-18T15:34:23.743489+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:23:53.146495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.745767+0000","last_change":"2024-03-18T15:34:21.755902+0000","last_active":"2024-03-18T15:34:23.745767+0000","last_peered":"2024-03-18T15:34:23.745767+0000","last_clean":"2024-03-18T15:34:23.745767+0000","last_became_active":"2024-03-18T15:34:21.755004+0000","last_became_peered":"2024-03-18T15:34:21.755004+0000","last_unstale":"2024-03-18T15:34:23.745767+0000","last_undegraded":"2024-03-18T15:34:23.745767+0000","last_fullsized":"2024-03-18T15:34:23.745767+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T01:57:48.185270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.742694+0000","last_change":"2024-03-18T15:34:22.793199+0000","last_active":"2024-03-18T15:34:23.742694+0000","last_peered":"2024-03-18T15:34:23.742694+0000","last_clean":"2024-03-18T15:34:23.742694+0000","last_became_active":"2024-03-18T15:34:22.792851+0000","last_became_peered":"2024-03-18T15:34:22.792851+0000","last_unstale":"2024-03-18T15:34:23.742694+0000","last_undegraded":"2024-03-18T15:34:23.742694+0000","last_fullsized":"2024-03-18T15:34:23.742694+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:25:24.865468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"29'2","reported_seq":16,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.749244+0000","last_change":"2024-03-18T15:34:21.769215+0000","last_active":"2024-03-18T15:34:23.749244+0000","last_peered":"2024-03-18T15:34:23.749244+0000","last_clean":"2024-03-18T15:34:23.749244+0000","last_became_active":"2024-03-18T15:34:21.768612+0000","last_became_peered":"2024-03-18T15:34:21.768612+0000","last_unstale":"2024-03-18T15:34:23.749244+0000","last_undegraded":"2024-03-18T15:34:23.749244+0000","last_fullsized":"2024-03-18T15:34:23.749244+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:58:53.162823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.743038+0000","last_change":"2024-03-18T15:34:22.768133+0000","last_active":"2024-03-18T15:34:23.743038+0000","last_peered":"2024-03-18T15:34:23.743038+0000","last_clean":"2024-03-18T15:34:23.743038+0000","last_became_active":"2024-03-18T15:34:22.767842+0000","last_became_peered":"2024-03-18T15:34:22.767842+0000","last_unstale":"2024-03-18T15:34:23.743038+0000","last_undegraded":"2024-03-18T15:34:23.743038+0000","last_fullsized":"2024-03-18T15:34:23.743038+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:17:46.800698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:24.259801+0000","last_change":"2024-03-18T15:34:21.772985+0000","last_active":"2024-03-18T15:34:24.259801+0000","last_peered":"2024-03-18T15:34:24.259801+0000","last_clean":"2024-03-18T15:34:24.259801+0000","last_became_active":"2024-03-18T15:34:21.772257+0000","last_became_peered":"2024-03-18T15:34:21.772257+0000","last_unstale":"2024-03-18T15:34:24.259801+0000","last_undegraded":"2024-03-18T15:34:24.259801+0000","last_fullsized":"2024-03-18T15:34:24.259801+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T15:57:42.258826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.742435+0000","last_change":"2024-03-18T15:34:22.752889+0000","last_active":"2024-03-18T15:34:23.742435+0000","last_peered":"2024-03-18T15:34:23.742435+0000","last_clean":"2024-03-18T15:34:23.742435+0000","last_became_active":"2024-03-18T15:34:22.752637+0000","last_became_peered":"2024-03-18T15:34:22.752637+0000","last_unstale":"2024-03-18T15:34:23.742435+0000","last_undegraded":"2024-03-18T15:34:23.742435+0000","last_fullsized":"2024-03-18T15:34:23.742435+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:17:15.098291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.743111+0000","last_change":"2024-03-18T15:34:21.755841+0000","last_active":"2024-03-18T15:34:23.743111+0000","last_peered":"2024-03-18T15:34:23.743111+0000","last_clean":"2024-03-18T15:34:23.743111+0000","last_became_active":"2024-03-18T15:34:21.755086+0000","last_became_peered":"2024-03-18T15:34:21.755086+0000","last_unstale":"2024-03-18T15:34:23.743111+0000","last_undegraded":"2024-03-18T15:34:23.743111+0000","last_fullsized":"2024-03-18T15:34:23.743111+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T18:52:15.262294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.743363+0000","last_change":"2024-03-18T15:34:22.785770+0000","last_active":"2024-03-18T15:34:23.743363+0000","last_peered":"2024-03-18T15:34:23.743363+0000","last_clean":"2024-03-18T15:34:23.743363+0000","last_became_active":"2024-03-18T15:34:22.784901+0000","last_became_peered":"2024-03-18T15:34:22.784901+0000","last_unstale":"2024-03-18T15:34:23.743363+0000","last_undegraded":"2024-03-18T15:34:23.743363+0000","last_fullsized":"2024-03-18T15:34:23.743363+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T22:33:48.332707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:24.173006+0000","last_change":"2024-03-18T15:34:21.772376+0000","last_active":"2024-03-18T15:34:24.173006+0000","last_peered":"2024-03-18T15:34:24.173006+0000","last_clean":"2024-03-18T15:34:24.173006+0000","last_became_active":"2024-03-18T15:34:21.771414+0000","last_became_peered":"2024-03-18T15:34:21.771414+0000","last_unstale":"2024-03-18T15:34:24.173006+0000","last_undegraded":"2024-03-18T15:34:24.173006+0000","last_fullsized":"2024-03-18T15:34:24.173006+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:12:12.866001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.743547+0000","last_change":"2024-03-18T15:34:22.778725+0000","last_active":"2024-03-18T15:34:23.743547+0000","last_peered":"2024-03-18T15:34:23.743547+0000","last_clean":"2024-03-18T15:34:23.743547+0000","last_became_active":"2024-03-18T15:34:22.778287+0000","last_became_peered":"2024-03-18T15:34:22.778287+0000","last_unstale":"2024-03-18T15:34:23.743547+0000","last_undegraded":"2024-03-18T15:34:23.743547+0000","last_fullsized":"2024-03-18T15:34:23.743547+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T15:57:43.268841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"29'2","reported_seq":16,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.752602+0000","last_change":"2024-03-18T15:34:21.777078+0000","last_active":"2024-03-18T15:34:23.752602+0000","last_peered":"2024-03-18T15:34:23.752602+0000","last_clean":"2024-03-18T15:34:23.752602+0000","last_became_active":"2024-03-18T15:34:21.776581+0000","last_became_peered":"2024-03-18T15:34:21.776581+0000","last_unstale":"2024-03-18T15:34:23.752602+0000","last_undegraded":"2024-03-18T15:34:23.752602+0000","last_fullsized":"2024-03-18T15:34:23.752602+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:56:42.710235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:24.173990+0000","last_change":"2024-03-18T15:34:22.776874+0000","last_active":"2024-03-18T15:34:24.173990+0000","last_peered":"2024-03-18T15:34:24.173990+0000","last_clean":"2024-03-18T15:34:24.173990+0000","last_became_active":"2024-03-18T15:34:22.776134+0000","last_became_peered":"2024-03-18T15:34:22.776134+0000","last_unstale":"2024-03-18T15:34:24.173990+0000","last_undegraded":"2024-03-18T15:34:24.173990+0000","last_fullsized":"2024-03-18T15:34:24.173990+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:42:50.876486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"30'1","reported_seq":15,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.750927+0000","last_change":"2024-03-18T15:34:21.769470+0000","last_active":"2024-03-18T15:34:23.750927+0000","last_peered":"2024-03-18T15:34:23.750927+0000","last_clean":"2024-03-18T15:34:23.750927+0000","last_became_active":"2024-03-18T15:34:21.769184+0000","last_became_peered":"2024-03-18T15:34:21.769184+0000","last_unstale":"2024-03-18T15:34:23.750927+0000","last_undegraded":"2024-03-18T15:34:23.750927+0000","last_fullsized":"2024-03-18T15:34:23.750927+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:34:09.363855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.749071+0000","last_change":"2024-03-18T15:34:22.765932+0000","last_active":"2024-03-18T15:34:23.749071+0000","last_peered":"2024-03-18T15:34:23.749071+0000","last_clean":"2024-03-18T15:34:23.749071+0000","last_became_active":"2024-03-18T15:34:22.765701+0000","last_became_peered":"2024-03-18T15:34:22.765701+0000","last_unstale":"2024-03-18T15:34:23.749071+0000","last_undegraded":"2024-03-18T15:34:23.749071+0000","last_fullsized":"2024-03-18T15:34:23.749071+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:02:26.456145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.752247+0000","last_change":"2024-03-18T15:34:21.763930+0000","last_active":"2024-03-18T15:34:23.752247+0000","last_peered":"2024-03-18T15:34:23.752247+0000","last_clean":"2024-03-18T15:34:23.752247+0000","last_became_active":"2024-03-18T15:34:21.763572+0000","last_became_peered":"2024-03-18T15:34:21.763572+0000","last_unstale":"2024-03-18T15:34:23.752247+0000","last_undegraded":"2024-03-18T15:34:23.752247+0000","last_fullsized":"2024-03-18T15:34:23.752247+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:49:56.976995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:24.259580+0000","last_change":"2024-03-18T15:34:22.783971+0000","last_active":"2024-03-18T15:34:24.259580+0000","last_peered":"2024-03-18T15:34:24.259580+0000","last_clean":"2024-03-18T15:34:24.259580+0000","last_became_active":"2024-03-18T15:34:22.783337+0000","last_became_peered":"2024-03-18T15:34:22.783337+0000","last_unstale":"2024-03-18T15:34:24.259580+0000","last_undegraded":"2024-03-18T15:34:24.259580+0000","last_fullsized":"2024-03-18T15:34:24.259580+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T20:25:36.746360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.749387+0000","last_change":"2024-03-18T15:34:21.777115+0000","last_active":"2024-03-18T15:34:23.749387+0000","last_peered":"2024-03-18T15:34:23.749387+0000","last_clean":"2024-03-18T15:34:23.749387+0000","last_became_active":"2024-03-18T15:34:21.776733+0000","last_became_peered":"2024-03-18T15:34:21.776733+0000","last_unstale":"2024-03-18T15:34:23.749387+0000","last_undegraded":"2024-03-18T15:34:23.749387+0000","last_fullsized":"2024-03-18T15:34:23.749387+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:01:19.496315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.749248+0000","last_change":"2024-03-18T15:34:22.785592+0000","last_active":"2024-03-18T15:34:23.749248+0000","last_peered":"2024-03-18T15:34:23.749248+0000","last_clean":"2024-03-18T15:34:23.749248+0000","last_became_active":"2024-03-18T15:34:22.785325+0000","last_became_peered":"2024-03-18T15:34:22.785325+0000","last_unstale":"2024-03-18T15:34:23.749248+0000","last_undegraded":"2024-03-18T15:34:23.749248+0000","last_fullsized":"2024-03-18T15:34:23.749248+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:03:09.703931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.751583+0000","last_change":"2024-03-18T15:34:21.769233+0000","last_active":"2024-03-18T15:34:23.751583+0000","last_peered":"2024-03-18T15:34:23.751583+0000","last_clean":"2024-03-18T15:34:23.751583+0000","last_became_active":"2024-03-18T15:34:21.768820+0000","last_became_peered":"2024-03-18T15:34:21.768820+0000","last_unstale":"2024-03-18T15:34:23.751583+0000","last_undegraded":"2024-03-18T15:34:23.751583+0000","last_fullsized":"2024-03-18T15:34:23.751583+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:54:04.203613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-18T15:34:22.781336+0000","last_change":"2024-03-18T15:34:22.768345+0000","last_active":"2024-03-18T15:34:22.781336+0000","last_peered":"2024-03-18T15:34:22.781336+0000","last_clean":"2024-03-18T15:34:22.781336+0000","last_became_active":"2024-03-18T15:34:22.767943+0000","last_became_peered":"2024-03-18T15:34:22.767943+0000","last_unstale":"2024-03-18T15:34:22.781336+0000","last_undegraded":"2024-03-18T15:34:22.781336+0000","last_fullsized":"2024-03-18T15:34:22.781336+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T23:21:26.063255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.743266+0000","last_change":"2024-03-18T15:34:21.767890+0000","last_active":"2024-03-18T15:34:23.743266+0000","last_peered":"2024-03-18T15:34:23.743266+0000","last_clean":"2024-03-18T15:34:23.743266+0000","last_became_active":"2024-03-18T15:34:21.767243+0000","last_became_peered":"2024-03-18T15:34:21.767243+0000","last_unstale":"2024-03-18T15:34:23.743266+0000","last_undegraded":"2024-03-18T15:34:23.743266+0000","last_fullsized":"2024-03-18T15:34:23.743266+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T00:10:40.586174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.742865+0000","last_change":"2024-03-18T15:34:22.792199+0000","last_active":"2024-03-18T15:34:23.742865+0000","last_peered":"2024-03-18T15:34:23.742865+0000","last_clean":"2024-03-18T15:34:23.742865+0000","last_became_active":"2024-03-18T15:34:22.791583+0000","last_became_peered":"2024-03-18T15:34:22.791583+0000","last_unstale":"2024-03-18T15:34:23.742865+0000","last_undegraded":"2024-03-18T15:34:23.742865+0000","last_fullsized":"2024-03-18T15:34:23.742865+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T16:46:07.311590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-03-18T15:34:22.763729+0000","last_change":"2024-03-18T15:34:21.760360+0000","last_active":"2024-03-18T15:34:22.763729+0000","last_peered":"2024-03-18T15:34:22.763729+0000","last_clean":"2024-03-18T15:34:22.763729+0000","last_became_active":"2024-03-18T15:34:21.759919+0000","last_became_peered":"2024-03-18T15:34:21.759919+0000","last_unstale":"2024-03-18T15:34:22.763729+0000","last_undegraded":"2024-03-18T15:34:22.763729+0000","last_fullsized":"2024-03-18T15:34:22.763729+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T17:52:33.287817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.743445+0000","last_change":"2024-03-18T15:34:22.764598+0000","last_active":"2024-03-18T15:34:23.743445+0000","last_peered":"2024-03-18T15:34:23.743445+0000","last_clean":"2024-03-18T15:34:23.743445+0000","last_became_active":"2024-03-18T15:34:22.763695+0000","last_became_peered":"2024-03-18T15:34:22.763695+0000","last_unstale":"2024-03-18T15:34:23.743445+0000","last_undegraded":"2024-03-18T15:34:23.743445+0000","last_fullsized":"2024-03-18T15:34:23.743445+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:21.723958+0000","last_clean_scrub_stamp":"2024-03-18T15:34:21.723958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T03:17:38.633080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:24.259912+0000","last_change":"2024-03-18T15:34:21.778297+0000","last_active":"2024-03-18T15:34:24.259912+0000","last_peered":"2024-03-18T15:34:24.259912+0000","last_clean":"2024-03-18T15:34:24.259912+0000","last_became_active":"2024-03-18T15:34:21.774483+0000","last_became_peered":"2024-03-18T15:34:21.774483+0000","last_unstale":"2024-03-18T15:34:24.259912+0000","last_undegraded":"2024-03-18T15:34:24.259912+0000","last_fullsized":"2024-03-18T15:34:24.259912+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:34:20.713943+0000","last_clean_scrub_stamp":"2024-03-18T15:34:20.713943+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-20T02:24:51.220302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":188,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-18T15:34:23.745026+0000","last_change":"2024-03-18T15:33:30.766315+0000","last_active":"2024-03-18T15:34:23.745026+0000","last_peered":"2024-03-18T15:34:23.745026+0000","last_clean":"2024-03-18T15:34:23.745026+0000","last_became_active":"2024-03-18T15:33:30.763939+0000","last_became_peered":"2024-03-18T15:33:30.763939+0000","last_unstale":"2024-03-18T15:34:23.745026+0000","last_undegraded":"2024-03-18T15:34:23.745026+0000","last_fullsized":"2024-03-18T15:34:23.745026+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:29.229911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:29.229911+0000","last_clean_scrub_stamp":"2024-03-18T15:33:29.229911+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:10:21.120371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509455,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27336,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":323584,"data_stored":120849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509455,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":335872,"data_stored":127205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542159,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":925696,"data_stored":716991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542160,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":335872,"data_stored":126637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542158,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35668,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336172,"statfs":{"total":96636764160,"available":96600240128,"internally_reserved":0,"allocated":331776,"data_stored":121867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542158,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35652,"kb_used_data":312,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336188,"statfs":{"total":96636764160,"available":96600256512,"internally_reserved":0,"allocated":319488,"data_stored":120303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542158,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35668,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336172,"statfs":{"total":96636764160,"available":96600240128,"internally_reserved":0,"allocated":331776,"data_stored":121867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542157,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36232,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335608,"statfs":{"total":96636764160,"available":96599662592,"internally_reserved":0,"allocated":909312,"data_stored":710601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-18T15:34:27.107 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-18T15:34:27.107 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-18T15:34:27.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.272+0000 7fa236f7c640 1 Processor -- start 2024-03-18T15:34:27.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.272+0000 7fa236f7c640 1 -- start start 2024-03-18T15:34:27.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.272+0000 7fa236f7c640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa230145db0 0x7fa230150a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:27.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.272+0000 7fa236f7c640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2300b5e10 con 0x7fa2300727d0 2024-03-18T15:34:27.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.272+0000 7fa236f7c640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa230072bb0 con 0x7fa2300b4580 2024-03-18T15:34:27.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.272+0000 7fa236f7c640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fa2300b1770 con 0x7fa230145db0 2024-03-18T15:34:27.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.272+0000 7fa234d18640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fa2300727d0 0x7fa2300b5700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47062/0 (socket says 172.21.15.5:47062) 2024-03-18T15:34:27.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.272+0000 7fa234d18640 1 -- 172.21.15.5:0/2958377323 learned_addr learned my addr 172.21.15.5:0/2958377323 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:27.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.272+0000 7fa22f7fe640 1 -- 172.21.15.5:0/2958377323 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 429142892 0 0) 0x7fa2300b5e10 con 0x7fa2300727d0 2024-03-18T15:34:27.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.272+0000 7fa22f7fe640 1 -- 172.21.15.5:0/2958377323 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa218003530 con 0x7fa2300727d0 2024-03-18T15:34:27.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.272+0000 7fa235519640 1 --2- 172.21.15.5:0/2958377323 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa230145db0 0x7fa230150a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:27.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.272+0000 7fa22f7fe640 1 -- 172.21.15.5:0/2958377323 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1559695432 0 0) 0x7fa218003530 con 0x7fa2300727d0 2024-03-18T15:34:27.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.272+0000 7fa22f7fe640 1 -- 172.21.15.5:0/2958377323 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa230145db0 msgr2=0x7fa230150a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:27.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.272+0000 7fa22f7fe640 1 --2- 172.21.15.5:0/2958377323 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa230145db0 0x7fa230150a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:27.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.272+0000 7fa22f7fe640 1 -- 172.21.15.5:0/2958377323 >> v1:172.21.15.80:6789/0 conn(0x7fa2300b4580 legacy=0x7fa2301456a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:27.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.272+0000 7fa22f7fe640 1 -- 172.21.15.5:0/2958377323 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2300b1a50 con 0x7fa2300727d0 2024-03-18T15:34:27.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.272+0000 7fa22f7fe640 1 -- 172.21.15.5:0/2958377323 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3837549003 0 0) 0x7fa220001a00 con 0x7fa2300727d0 2024-03-18T15:34:27.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.272+0000 7fa22f7fe640 1 -- 172.21.15.5:0/2958377323 >> v1:172.21.15.5:6789/0 conn(0x7fa2300727d0 legacy=0x7fa2300b5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:27.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.272+0000 7fa22f7fe640 1 --2- 172.21.15.5:0/2958377323 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa218003ad0 0x7fa218003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:27.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.272+0000 7fa22f7fe640 1 --2- 172.21.15.5:0/2958377323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa218004810 0x7fa218004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:27.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.272+0000 7fa22f7fe640 1 --2- 172.21.15.5:0/2958377323 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa230145db0 0x7fa21800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:27.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.272+0000 7fa22f7fe640 1 -- 172.21.15.5:0/2958377323 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa2300b1770 con 0x7fa218004810 2024-03-18T15:34:27.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.272+0000 7fa22f7fe640 1 -- 172.21.15.5:0/2958377323 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa218005530 con 0x7fa218003ad0 2024-03-18T15:34:27.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.272+0000 7fa22f7fe640 1 -- 172.21.15.5:0/2958377323 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fa21800bf10 con 0x7fa230145db0 2024-03-18T15:34:27.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.272+0000 7fa22ffff640 1 --2- 172.21.15.5:0/2958377323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa218004810 0x7fa218004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:27.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.272+0000 7fa234d18640 1 --2- 172.21.15.5:0/2958377323 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa218003ad0 0x7fa218003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:27.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.272+0000 7fa235519640 1 --2- 172.21.15.5:0/2958377323 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa230145db0 0x7fa21800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:27.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.272+0000 7fa22ffff640 1 -- 172.21.15.5:0/2958377323 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa230145db0 msgr2=0x7fa21800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:27.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.272+0000 7fa22ffff640 1 --2- 172.21.15.5:0/2958377323 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa230145db0 0x7fa21800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:27.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.272+0000 7fa22ffff640 1 -- 172.21.15.5:0/2958377323 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa218003ad0 msgr2=0x7fa218003ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:27.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.272+0000 7fa22ffff640 1 --2- 172.21.15.5:0/2958377323 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa218003ad0 0x7fa218003ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:27.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.272+0000 7fa22ffff640 1 -- 172.21.15.5:0/2958377323 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa218003530 con 0x7fa218004810 2024-03-18T15:34:27.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.272+0000 7fa234d18640 1 --2- 172.21.15.5:0/2958377323 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa218003ad0 0x7fa218003ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:34:27.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.276+0000 7fa22ffff640 1 --2- 172.21.15.5:0/2958377323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa218004810 0x7fa218004c90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa224001780 tx=0x7fa22401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8abe94e8a14c5e91 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:27.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.276+0000 7fa22f7fe640 1 -- 172.21.15.5:0/2958377323 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2240013a0 con 0x7fa218004810 2024-03-18T15:34:27.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.276+0000 7fa22f7fe640 1 -- 172.21.15.5:0/2958377323 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2240052f0 con 0x7fa218004810 2024-03-18T15:34:27.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.276+0000 7fa22f7fe640 1 -- 172.21.15.5:0/2958377323 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa224003840 con 0x7fa218004810 2024-03-18T15:34:27.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.276+0000 7fa236f7c640 1 -- 172.21.15.5:0/2958377323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa218004810 msgr2=0x7fa218004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:27.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.276+0000 7fa236f7c640 1 --2- 172.21.15.5:0/2958377323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa218004810 0x7fa218004c90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa224001780 tx=0x7fa22401c420 comp rx=0 tx=0).stop 2024-03-18T15:34:27.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.276+0000 7fa235519640 1 -- 172.21.15.5:0/2958377323 reap_dead start 2024-03-18T15:34:27.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.276+0000 7fa236f7c640 1 -- 172.21.15.5:0/2958377323 shutdown_connections 2024-03-18T15:34:27.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.276+0000 7fa236f7c640 1 -- 172.21.15.5:0/2958377323 >> 172.21.15.5:0/2958377323 conn(0x7fa2300b6680 msgr2=0x7fa2300b6a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:27.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.276+0000 7fa236f7c640 1 -- 172.21.15.5:0/2958377323 shutdown_connections 2024-03-18T15:34:27.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.276+0000 7fa236f7c640 1 -- 172.21.15.5:0/2958377323 wait complete. 2024-03-18T15:34:27.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.280+0000 7fa236f7c640 1 Processor -- start 2024-03-18T15:34:27.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.280+0000 7fa236f7c640 1 -- start start 2024-03-18T15:34:27.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.280+0000 7fa236f7c640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa218004810 0x7fa2301384c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:27.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.280+0000 7fa236f7c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa2301337c0 0x7fa23013b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:27.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.280+0000 7fa236f7c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa23013b810 0x7fa2301e3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:27.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.280+0000 7fa236f7c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa2300b4020 con 0x7fa23013b810 2024-03-18T15:34:27.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.280+0000 7fa236f7c640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa2300b3ea0 con 0x7fa2301337c0 2024-03-18T15:34:27.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.280+0000 7fa236f7c640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fa230151260 con 0x7fa218004810 2024-03-18T15:34:27.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.280+0000 7fa235519640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa23013b810 0x7fa2301e3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:27.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.280+0000 7fa235519640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa23013b810 0x7fa2301e3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45100/0 (socket says 172.21.15.5:45100) 2024-03-18T15:34:27.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.280+0000 7fa235519640 1 -- 172.21.15.5:0/638819687 learned_addr learned my addr 172.21.15.5:0/638819687 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:27.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.280+0000 7fa234d18640 1 --2- 172.21.15.5:0/638819687 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa218004810 0x7fa2301384c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:27.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.280+0000 7fa22ffff640 1 --2- 172.21.15.5:0/638819687 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa2301337c0 0x7fa23013b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:27.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.280+0000 7fa235519640 1 -- 172.21.15.5:0/638819687 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa218004810 msgr2=0x7fa2301384c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:27.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.280+0000 7fa235519640 1 --2- 172.21.15.5:0/638819687 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa218004810 0x7fa2301384c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:27.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.280+0000 7fa235519640 1 -- 172.21.15.5:0/638819687 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa2301337c0 msgr2=0x7fa23013b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:27.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.280+0000 7fa235519640 1 --2- 172.21.15.5:0/638819687 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa2301337c0 0x7fa23013b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:27.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.280+0000 7fa235519640 1 -- 172.21.15.5:0/638819687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa23013bc50 con 0x7fa23013b810 2024-03-18T15:34:27.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.280+0000 7fa235519640 1 --2- 172.21.15.5:0/638819687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa23013b810 0x7fa2301e3740 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa2300b65e0 tx=0x7fa228011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:27.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.280+0000 7fa22d7fa640 1 -- 172.21.15.5:0/638819687 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa228019020 con 0x7fa23013b810 2024-03-18T15:34:27.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.280+0000 7fa22d7fa640 1 -- 172.21.15.5:0/638819687 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa228004240 con 0x7fa23013b810 2024-03-18T15:34:27.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.280+0000 7fa236f7c640 1 -- 172.21.15.5:0/638819687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa230133f80 con 0x7fa23013b810 2024-03-18T15:34:27.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.280+0000 7fa22d7fa640 1 -- 172.21.15.5:0/638819687 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa22800f7a0 con 0x7fa23013b810 2024-03-18T15:34:27.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.280+0000 7fa236f7c640 1 -- 172.21.15.5:0/638819687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa230133c40 con 0x7fa23013b810 2024-03-18T15:34:27.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.280+0000 7fa236f7c640 1 -- 172.21.15.5:0/638819687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa23004f550 con 0x7fa23013b810 2024-03-18T15:34:27.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.280+0000 7fa22d7fa640 1 -- 172.21.15.5:0/638819687 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fa22801c020 con 0x7fa23013b810 2024-03-18T15:34:27.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.284+0000 7fa22d7fa640 1 --2- 172.21.15.5:0/638819687 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa21c067660 0x7fa21c069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:27.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.284+0000 7fa234d18640 1 --2- 172.21.15.5:0/638819687 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa21c067660 0x7fa21c069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:27.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.284+0000 7fa22d7fa640 1 -- 172.21.15.5:0/638819687 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7fa228085c00 con 0x7fa23013b810 2024-03-18T15:34:27.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.284+0000 7fa234d18640 1 --2- 172.21.15.5:0/638819687 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa21c067660 0x7fa21c069b20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa220002240 tx=0x7fa220009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:27.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.288+0000 7fa22d7fa640 1 -- 172.21.15.5:0/638819687 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fa22808c080 con 0x7fa23013b810 2024-03-18T15:34:27.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.500+0000 7fa236f7c640 1 -- 172.21.15.5:0/638819687 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fa2300b2fd0 con 0x7fa23013b810 2024-03-18T15:34:27.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.504+0000 7fa22d7fa640 1 -- 172.21.15.5:0/638819687 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7fa228023020 con 0x7fa23013b810 2024-03-18T15:34:27.506 INFO:teuthology.orchestra.run.smithi005.stdout:[client.0] 2024-03-18T15:34:27.506 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQCeXvhlDBsBORAA9deuFWz05eGfJp1vj5Vlkg== 2024-03-18T15:34:27.506 INFO:teuthology.orchestra.run.smithi005.stdout: caps mds = "allow" 2024-03-18T15:34:27.506 INFO:teuthology.orchestra.run.smithi005.stdout: caps mon = "allow r" 2024-03-18T15:34:27.506 INFO:teuthology.orchestra.run.smithi005.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-18T15:34:27.507 INFO:teuthology.orchestra.run.smithi005.stderr:updated caps for client.0 2024-03-18T15:34:27.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.504+0000 7fa236f7c640 1 -- 172.21.15.5:0/638819687 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa21c067660 msgr2=0x7fa21c069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:27.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.504+0000 7fa236f7c640 1 --2- 172.21.15.5:0/638819687 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa21c067660 0x7fa21c069b20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa220002240 tx=0x7fa220009040 comp rx=0 tx=0).stop 2024-03-18T15:34:27.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.504+0000 7fa236f7c640 1 -- 172.21.15.5:0/638819687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa23013b810 msgr2=0x7fa2301e3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:27.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.504+0000 7fa236f7c640 1 --2- 172.21.15.5:0/638819687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa23013b810 0x7fa2301e3740 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa2300b65e0 tx=0x7fa228011cf0 comp rx=0 tx=0).stop 2024-03-18T15:34:27.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.504+0000 7fa236f7c640 1 -- 172.21.15.5:0/638819687 shutdown_connections 2024-03-18T15:34:27.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.504+0000 7fa236f7c640 1 --2- 172.21.15.5:0/638819687 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fa21c067660 0x7fa21c069b20 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:27.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.504+0000 7fa236f7c640 1 --2- 172.21.15.5:0/638819687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa23013b810 0x7fa2301e3740 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:27.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.504+0000 7fa236f7c640 1 --2- 172.21.15.5:0/638819687 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa2301337c0 0x7fa23013b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:27.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.504+0000 7fa236f7c640 1 --2- 172.21.15.5:0/638819687 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fa218004810 0x7fa2301384c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:27.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.504+0000 7fa236f7c640 1 -- 172.21.15.5:0/638819687 >> 172.21.15.5:0/638819687 conn(0x7fa2300b6680 msgr2=0x7fa2300b5700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:27.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.504+0000 7fa236f7c640 1 -- 172.21.15.5:0/638819687 shutdown_connections 2024-03-18T15:34:27.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.504+0000 7fa236f7c640 1 -- 172.21.15.5:0/638819687 wait complete. 2024-03-18T15:34:27.524 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:34:27.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.628+0000 7efc98ffb640 1 Processor -- start 2024-03-18T15:34:27.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.632+0000 7efc98ffb640 1 -- start start 2024-03-18T15:34:27.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.632+0000 7efc98ffb640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efc940b4580 0x7efc941456a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:27.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.632+0000 7efc98ffb640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7efc94145be0 0x7efc9414fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:27.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.632+0000 7efc98ffb640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc940b5e10 con 0x7efc940727d0 2024-03-18T15:34:27.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.632+0000 7efc98ffb640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7efc940b1770 con 0x7efc94145be0 2024-03-18T15:34:27.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.632+0000 7efc98ffb640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7efc94072bb0 con 0x7efc940b4580 2024-03-18T15:34:27.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.632+0000 7efc9259c640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7efc940727d0 0x7efc940b5700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:47078/0 (socket says 172.21.15.5:47078) 2024-03-18T15:34:27.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.632+0000 7efc9259c640 1 -- 172.21.15.5:0/2336473030 learned_addr learned my addr 172.21.15.5:0/2336473030 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:27.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.632+0000 7efc91d9b640 1 -- 172.21.15.5:0/2336473030 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2016167831 0 0) 0x7efc940b5e10 con 0x7efc940727d0 2024-03-18T15:34:27.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.632+0000 7efc91d9b640 1 -- 172.21.15.5:0/2336473030 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc80003530 con 0x7efc940727d0 2024-03-18T15:34:27.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.632+0000 7efc89d9b640 1 --2- 172.21.15.5:0/2336473030 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efc940b4580 0x7efc941456a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:27.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.632+0000 7efc92d9d640 1 --2- 172.21.15.5:0/2336473030 >> v2:172.21.15.80:3300/0 conn(0x7efc94145be0 0x7efc9414fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:27.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.632+0000 7efc91d9b640 1 -- 172.21.15.5:0/2336473030 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3167935004 0 0) 0x7efc80003530 con 0x7efc940727d0 2024-03-18T15:34:27.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.632+0000 7efc91d9b640 1 -- 172.21.15.5:0/2336473030 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efc940b4580 msgr2=0x7efc941456a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:27.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.632+0000 7efc91d9b640 1 --2- 172.21.15.5:0/2336473030 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efc940b4580 0x7efc941456a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:27.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.632+0000 7efc91d9b640 1 -- 172.21.15.5:0/2336473030 >> v2:172.21.15.80:3300/0 conn(0x7efc94145be0 msgr2=0x7efc9414fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:27.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.632+0000 7efc91d9b640 1 --2- 172.21.15.5:0/2336473030 >> v2:172.21.15.80:3300/0 conn(0x7efc94145be0 0x7efc9414fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:27.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.632+0000 7efc91d9b640 1 -- 172.21.15.5:0/2336473030 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc94146030 con 0x7efc940727d0 2024-03-18T15:34:27.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.632+0000 7efc91d9b640 1 -- 172.21.15.5:0/2336473030 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3837549003 0 0) 0x7efc84001680 con 0x7efc940727d0 2024-03-18T15:34:27.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.632+0000 7efc91d9b640 1 -- 172.21.15.5:0/2336473030 >> v1:172.21.15.5:6789/0 conn(0x7efc940727d0 legacy=0x7efc940b5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:27.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.632+0000 7efc91d9b640 1 --2- 172.21.15.5:0/2336473030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc80003aa0 0x7efc80003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:27.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.632+0000 7efc91d9b640 1 --2- 172.21.15.5:0/2336473030 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc80004810 0x7efc80004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:27.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.632+0000 7efc91d9b640 1 --2- 172.21.15.5:0/2336473030 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efc80005220 0x7efc8000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:27.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.632+0000 7efc91d9b640 1 -- 172.21.15.5:0/2336473030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efc940b1770 con 0x7efc80003aa0 2024-03-18T15:34:27.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.632+0000 7efc91d9b640 1 -- 172.21.15.5:0/2336473030 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7efc94072bb0 con 0x7efc80004810 2024-03-18T15:34:27.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.632+0000 7efc91d9b640 1 -- 172.21.15.5:0/2336473030 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7efc8000bfe0 con 0x7efc80005220 2024-03-18T15:34:27.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.632+0000 7efc9259c640 1 --2- 172.21.15.5:0/2336473030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc80003aa0 0x7efc80003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:27.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.632+0000 7efc92d9d640 1 --2- 172.21.15.5:0/2336473030 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efc80005220 0x7efc8000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:27.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.632+0000 7efc89d9b640 1 --2- 172.21.15.5:0/2336473030 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc80004810 0x7efc80004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:27.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.632+0000 7efc9259c640 1 -- 172.21.15.5:0/2336473030 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efc80005220 msgr2=0x7efc8000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:27.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.632+0000 7efc9259c640 1 --2- 172.21.15.5:0/2336473030 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efc80005220 0x7efc8000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:27.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.632+0000 7efc9259c640 1 -- 172.21.15.5:0/2336473030 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc80004810 msgr2=0x7efc80004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:27.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.632+0000 7efc9259c640 1 --2- 172.21.15.5:0/2336473030 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc80004810 0x7efc80004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:27.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.632+0000 7efc9259c640 1 -- 172.21.15.5:0/2336473030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc80003530 con 0x7efc80003aa0 2024-03-18T15:34:27.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.632+0000 7efc89d9b640 1 --2- 172.21.15.5:0/2336473030 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc80004810 0x7efc80004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:27.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.636+0000 7efc9259c640 1 --2- 172.21.15.5:0/2336473030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc80003aa0 0x7efc80003ec0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7efc8400c210 tx=0x7efc8401cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=742d14f326c226a7 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:27.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.636+0000 7efc91d9b640 1 -- 172.21.15.5:0/2336473030 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc84007d60 con 0x7efc80003aa0 2024-03-18T15:34:27.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.636+0000 7efc91d9b640 1 -- 172.21.15.5:0/2336473030 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc840a9020 con 0x7efc80003aa0 2024-03-18T15:34:27.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.636+0000 7efc91d9b640 1 -- 172.21.15.5:0/2336473030 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc84005b20 con 0x7efc80003aa0 2024-03-18T15:34:27.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.636+0000 7efc98ffb640 1 -- 172.21.15.5:0/2336473030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc80003aa0 msgr2=0x7efc80003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:27.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.636+0000 7efc98ffb640 1 --2- 172.21.15.5:0/2336473030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc80003aa0 0x7efc80003ec0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7efc8400c210 tx=0x7efc8401cbe0 comp rx=0 tx=0).stop 2024-03-18T15:34:27.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.636+0000 7efc92d9d640 1 -- 172.21.15.5:0/2336473030 reap_dead start 2024-03-18T15:34:27.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.636+0000 7efc98ffb640 1 -- 172.21.15.5:0/2336473030 shutdown_connections 2024-03-18T15:34:27.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.636+0000 7efc98ffb640 1 -- 172.21.15.5:0/2336473030 >> 172.21.15.5:0/2336473030 conn(0x7efc940b6680 msgr2=0x7efc940b6a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:27.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.636+0000 7efc98ffb640 1 -- 172.21.15.5:0/2336473030 shutdown_connections 2024-03-18T15:34:27.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.636+0000 7efc98ffb640 1 -- 172.21.15.5:0/2336473030 wait complete. 2024-03-18T15:34:27.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.636+0000 7efc98ffb640 1 Processor -- start 2024-03-18T15:34:27.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.636+0000 7efc98ffb640 1 -- start start 2024-03-18T15:34:27.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.636+0000 7efc98ffb640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efc80003aa0 0x7efc940afbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:27.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.636+0000 7efc98ffb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc9414afc0 0x7efc941de070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:27.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.636+0000 7efc98ffb640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc9414b440 0x7efc941de5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:27.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.636+0000 7efc98ffb640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efc940b3ea0 con 0x7efc9414afc0 2024-03-18T15:34:27.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.636+0000 7efc98ffb640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7efc941450e0 con 0x7efc9414b440 2024-03-18T15:34:27.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.636+0000 7efc98ffb640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7efc940b4020 con 0x7efc80003aa0 2024-03-18T15:34:27.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.636+0000 7efc89d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc9414afc0 0x7efc941de070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:27.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.636+0000 7efc89d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc9414afc0 0x7efc941de070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45122/0 (socket says 172.21.15.5:45122) 2024-03-18T15:34:27.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.636+0000 7efc89d9b640 1 -- 172.21.15.5:0/715590734 learned_addr learned my addr 172.21.15.5:0/715590734 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:27.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.636+0000 7efc92d9d640 1 --2- 172.21.15.5:0/715590734 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc9414b440 0x7efc941de5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:27.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.636+0000 7efc9259c640 1 --2- 172.21.15.5:0/715590734 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efc80003aa0 0x7efc940afbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:27.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.636+0000 7efc89d9b640 1 -- 172.21.15.5:0/715590734 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efc80003aa0 msgr2=0x7efc940afbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:27.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.636+0000 7efc89d9b640 1 --2- 172.21.15.5:0/715590734 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efc80003aa0 0x7efc940afbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:27.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.636+0000 7efc89d9b640 1 -- 172.21.15.5:0/715590734 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc9414b440 msgr2=0x7efc941de5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:27.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.636+0000 7efc89d9b640 1 --2- 172.21.15.5:0/715590734 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc9414b440 0x7efc941de5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:27.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.636+0000 7efc89d9b640 1 -- 172.21.15.5:0/715590734 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc941deaf0 con 0x7efc9414afc0 2024-03-18T15:34:27.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.636+0000 7efc92d9d640 1 --2- 172.21.15.5:0/715590734 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc9414b440 0x7efc941de5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:34:27.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.636+0000 7efc9259c640 1 --2- 172.21.15.5:0/715590734 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efc80003aa0 0x7efc940afbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:34:27.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.636+0000 7efc89d9b640 1 --2- 172.21.15.5:0/715590734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc9414afc0 0x7efc941de070 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7efc7c00e810 tx=0x7efc7c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:27.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.636+0000 7efc8b7fe640 1 -- 172.21.15.5:0/715590734 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7c017070 con 0x7efc9414afc0 2024-03-18T15:34:27.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.636+0000 7efc8b7fe640 1 -- 172.21.15.5:0/715590734 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc7c00c6d0 con 0x7efc9414afc0 2024-03-18T15:34:27.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.636+0000 7efc98ffb640 1 -- 172.21.15.5:0/715590734 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc940a9920 con 0x7efc9414afc0 2024-03-18T15:34:27.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.636+0000 7efc8b7fe640 1 -- 172.21.15.5:0/715590734 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7c00f040 con 0x7efc9414afc0 2024-03-18T15:34:27.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.636+0000 7efc98ffb640 1 -- 172.21.15.5:0/715590734 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc940ab470 con 0x7efc9414afc0 2024-03-18T15:34:27.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.640+0000 7efc98ffb640 1 -- 172.21.15.5:0/715590734 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc9404f550 con 0x7efc9414afc0 2024-03-18T15:34:27.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.640+0000 7efc8b7fe640 1 -- 172.21.15.5:0/715590734 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7efc7c01a070 con 0x7efc9414afc0 2024-03-18T15:34:27.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.640+0000 7efc8b7fe640 1 --2- 172.21.15.5:0/715590734 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7efc74067660 0x7efc74069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:27.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.640+0000 7efc8b7fe640 1 -- 172.21.15.5:0/715590734 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7efc7c085420 con 0x7efc9414afc0 2024-03-18T15:34:27.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.640+0000 7efc9259c640 1 --2- 172.21.15.5:0/715590734 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7efc74067660 0x7efc74069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:27.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.640+0000 7efc9259c640 1 --2- 172.21.15.5:0/715590734 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7efc74067660 0x7efc74069b20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7efc840aa000 tx=0x7efc84000ff0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:27.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.644+0000 7efc8b7fe640 1 -- 172.21.15.5:0/715590734 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7efc7c056090 con 0x7efc9414afc0 2024-03-18T15:34:27.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.836+0000 7efc98ffb640 1 -- 172.21.15.5:0/715590734 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc94142b20 con 0x7efc9414afc0 2024-03-18T15:34:27.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.836+0000 7efc8b7fe640 1 -- 172.21.15.5:0/715590734 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+3528 (secure 0 0 0) 0x7efc7c055990 con 0x7efc9414afc0 2024-03-18T15:34:27.838 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:34:27.839 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4619,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/2921781388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":2921781388},{"type":"v1","addr":"172.21.15.80:6833","nonce":2921781388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-18T15:34:22.108585+0000","modified":"2024-03-18T15:34:25.110609+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4968},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4968":{"gid":4968,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6835/2235154713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2235154713},{"type":"v1","addr":"172.21.15.5:6835","nonce":2235154713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4968,"qdb_cluster":[4968]},"id":2}]} 2024-03-18T15:34:27.839 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 16 2024-03-18T15:34:27.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.836+0000 7efc98ffb640 1 -- 172.21.15.5:0/715590734 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7efc74067660 msgr2=0x7efc74069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:27.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.836+0000 7efc98ffb640 1 --2- 172.21.15.5:0/715590734 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7efc74067660 0x7efc74069b20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7efc840aa000 tx=0x7efc84000ff0 comp rx=0 tx=0).stop 2024-03-18T15:34:27.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.836+0000 7efc98ffb640 1 -- 172.21.15.5:0/715590734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc9414afc0 msgr2=0x7efc941de070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:27.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.836+0000 7efc98ffb640 1 --2- 172.21.15.5:0/715590734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc9414afc0 0x7efc941de070 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7efc7c00e810 tx=0x7efc7c00ecd0 comp rx=0 tx=0).stop 2024-03-18T15:34:27.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.836+0000 7efc98ffb640 1 -- 172.21.15.5:0/715590734 shutdown_connections 2024-03-18T15:34:27.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.836+0000 7efc98ffb640 1 --2- 172.21.15.5:0/715590734 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7efc74067660 0x7efc74069b20 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:27.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.836+0000 7efc98ffb640 1 --2- 172.21.15.5:0/715590734 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc9414b440 0x7efc941de5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:27.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.836+0000 7efc98ffb640 1 --2- 172.21.15.5:0/715590734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc9414afc0 0x7efc941de070 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:27.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.836+0000 7efc98ffb640 1 --2- 172.21.15.5:0/715590734 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7efc80003aa0 0x7efc940afbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:27.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.836+0000 7efc98ffb640 1 -- 172.21.15.5:0/715590734 >> 172.21.15.5:0/715590734 conn(0x7efc940b6680 msgr2=0x7efc94143050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:27.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.836+0000 7efc98ffb640 1 -- 172.21.15.5:0/715590734 shutdown_connections 2024-03-18T15:34:27.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:27.836+0000 7efc98ffb640 1 -- 172.21.15.5:0/715590734 wait complete. 2024-03-18T15:34:28.057 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-18T15:34:22.108585+0000', 'modified': '2024-03-18T15:34:25.110609+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4968}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4968': {'gid': 4968, 'name': 'a', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.5:6835/2235154713', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6834', 'nonce': 2235154713}, {'type': 'v1', 'addr': '172.21.15.5:6835', 'nonce': 2235154713}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4968, 'qdb_cluster': [4968]} 2024-03-18T15:34:28.058 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-18T15:34:28.058 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-18T15:34:28.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.176+0000 7f5634718640 1 Processor -- start 2024-03-18T15:34:28.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.176+0000 7f5634718640 1 -- start start 2024-03-18T15:34:28.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.176+0000 7f5634718640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f562c14b030 0x7f562c140ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:28.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.176+0000 7f5634718640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f562c141430 0x7f562c151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:28.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.176+0000 7f5634718640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f562c0b9d50 con 0x7f562c0b5710 2024-03-18T15:34:28.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.176+0000 7f5634718640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f562c0b4320 con 0x7f562c141430 2024-03-18T15:34:28.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.176+0000 7f5634718640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f562c0b4570 con 0x7f562c14b030 2024-03-18T15:34:28.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.176+0000 7f5632cb5640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f562c141430 0x7f562c151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:28.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.176+0000 7f5632cb5640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f562c141430 0x7f562c151cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45124/0 (socket says 172.21.15.5:45124) 2024-03-18T15:34:28.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.176+0000 7f5632cb5640 1 -- 172.21.15.5:0/2057800494 learned_addr learned my addr 172.21.15.5:0/2057800494 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:28.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.176+0000 7f5631cb3640 1 --2- 172.21.15.5:0/2057800494 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f562c14b030 0x7f562c140ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:28.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.176+0000 7f5632cb5640 1 -- 172.21.15.5:0/2057800494 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f562c14b030 msgr2=0x7f562c140ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:28.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.176+0000 7f5632cb5640 1 --2- 172.21.15.5:0/2057800494 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f562c14b030 0x7f562c140ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:28.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.176+0000 7f5632cb5640 1 -- 172.21.15.5:0/2057800494 >> v1:172.21.15.80:6789/0 conn(0x7f562c0b5710 legacy=0x7f562c14a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:28.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.176+0000 7f5632cb5640 1 -- 172.21.15.5:0/2057800494 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f562c0b4c30 con 0x7f562c141430 2024-03-18T15:34:28.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.180+0000 7f5632cb5640 1 --2- 172.21.15.5:0/2057800494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f562c141430 0x7f562c151cb0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5624009e80 tx=0x7f562401c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73ef28fde51d9296 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:28.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.180+0000 7f56314b2640 1 -- 172.21.15.5:0/2057800494 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56240a55b0 con 0x7f562c141430 2024-03-18T15:34:28.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.180+0000 7f56314b2640 1 -- 172.21.15.5:0/2057800494 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56240a5750 con 0x7f562c141430 2024-03-18T15:34:28.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.180+0000 7f56314b2640 1 -- 172.21.15.5:0/2057800494 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56240a5b20 con 0x7f562c141430 2024-03-18T15:34:28.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.180+0000 7f5634718640 1 -- 172.21.15.5:0/2057800494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f562c141430 msgr2=0x7f562c151cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:28.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.180+0000 7f5634718640 1 --2- 172.21.15.5:0/2057800494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f562c141430 0x7f562c151cb0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5624009e80 tx=0x7f562401c8f0 comp rx=0 tx=0).stop 2024-03-18T15:34:28.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.180+0000 7f5634718640 1 -- 172.21.15.5:0/2057800494 shutdown_connections 2024-03-18T15:34:28.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.180+0000 7f5634718640 1 --2- 172.21.15.5:0/2057800494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f562c141430 0x7f562c151cb0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:28.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.180+0000 7f5634718640 1 --2- 172.21.15.5:0/2057800494 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f562c14b030 0x7f562c140ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:28.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.180+0000 7f5634718640 1 -- 172.21.15.5:0/2057800494 >> 172.21.15.5:0/2057800494 conn(0x7f562c0673c0 msgr2=0x7f562c0677d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:28.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.180+0000 7f5634718640 1 -- 172.21.15.5:0/2057800494 shutdown_connections 2024-03-18T15:34:28.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.180+0000 7f5634718640 1 -- 172.21.15.5:0/2057800494 wait complete. 2024-03-18T15:34:28.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.184+0000 7f5634718640 1 Processor -- start 2024-03-18T15:34:28.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.184+0000 7f5634718640 1 -- start start 2024-03-18T15:34:28.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.184+0000 7f5634718640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f562c0b5710 0x7f562c1361a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:28.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.184+0000 7f5634718640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f562c141430 0x7f562c1dff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:28.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.184+0000 7f5634718640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f562c14b030 0x7f562c1e0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:28.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.184+0000 7f5634718640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f562c0b5260 con 0x7f562c0b5710 2024-03-18T15:34:28.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.184+0000 7f5634718640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f562c0b50e0 con 0x7f562c14b030 2024-03-18T15:34:28.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.184+0000 7f5634718640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f562c154a10 con 0x7f562c141430 2024-03-18T15:34:28.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.184+0000 7f56324b4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f562c0b5710 0x7f562c1361a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:28.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.184+0000 7f56324b4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f562c0b5710 0x7f562c1361a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45138/0 (socket says 172.21.15.5:45138) 2024-03-18T15:34:28.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.184+0000 7f56324b4640 1 -- 172.21.15.5:0/3754524275 learned_addr learned my addr 172.21.15.5:0/3754524275 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:28.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.184+0000 7f5631cb3640 1 --2- 172.21.15.5:0/3754524275 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f562c141430 0x7f562c1dff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:28.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.184+0000 7f5632cb5640 1 --2- 172.21.15.5:0/3754524275 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f562c14b030 0x7f562c1e0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:28.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.184+0000 7f56324b4640 1 -- 172.21.15.5:0/3754524275 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f562c141430 msgr2=0x7f562c1dff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:28.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.184+0000 7f56324b4640 1 --2- 172.21.15.5:0/3754524275 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f562c141430 0x7f562c1dff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:28.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.184+0000 7f56324b4640 1 -- 172.21.15.5:0/3754524275 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f562c14b030 msgr2=0x7f562c1e0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:28.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.184+0000 7f56324b4640 1 --2- 172.21.15.5:0/3754524275 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f562c14b030 0x7f562c1e0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:28.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.184+0000 7f56324b4640 1 -- 172.21.15.5:0/3754524275 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f562c13a490 con 0x7f562c0b5710 2024-03-18T15:34:28.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.184+0000 7f56324b4640 1 --2- 172.21.15.5:0/3754524275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f562c0b5710 0x7f562c1361a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f562800c420 tx=0x7f562800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:28.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.184+0000 7f561effd640 1 -- 172.21.15.5:0/3754524275 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56280052d0 con 0x7f562c0b5710 2024-03-18T15:34:28.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.184+0000 7f561effd640 1 -- 172.21.15.5:0/3754524275 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5628005470 con 0x7f562c0b5710 2024-03-18T15:34:28.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.184+0000 7f5634718640 1 -- 172.21.15.5:0/3754524275 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f562c13a770 con 0x7f562c0b5710 2024-03-18T15:34:28.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.184+0000 7f561effd640 1 -- 172.21.15.5:0/3754524275 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5628007260 con 0x7f562c0b5710 2024-03-18T15:34:28.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.184+0000 7f5634718640 1 -- 172.21.15.5:0/3754524275 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f562c13ce80 con 0x7f562c0b5710 2024-03-18T15:34:28.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.184+0000 7f561effd640 1 -- 172.21.15.5:0/3754524275 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f5628007400 con 0x7f562c0b5710 2024-03-18T15:34:28.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.188+0000 7f561effd640 1 --2- 172.21.15.5:0/3754524275 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5610067660 0x7f5610069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:28.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.188+0000 7f561effd640 1 -- 172.21.15.5:0/3754524275 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7f5628084eb0 con 0x7f562c0b5710 2024-03-18T15:34:28.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.188+0000 7f5631cb3640 1 --2- 172.21.15.5:0/3754524275 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5610067660 0x7f5610069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:28.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.188+0000 7f5631cb3640 1 --2- 172.21.15.5:0/3754524275 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5610067660 0x7f5610069b20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f562c068c20 tx=0x7f5620008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:28.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.188+0000 7f563789a000 1 Processor -- start 2024-03-18T15:34:28.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.188+0000 7f563789a000 1 -- start start 2024-03-18T15:34:28.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.188+0000 7f563789a000 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563988ef94a0 0x563988ef9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:28.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.188+0000 7f563789a000 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563988ef9dc0 0x563988f24180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:28.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.188+0000 7f563789a000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563988f246c0 0x563988f24aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:28.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.188+0000 7f563789a000 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563988d42540 con 0x563988ef94a0 2024-03-18T15:34:28.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.188+0000 7f563789a000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x563988cc9b00 con 0x563988f246c0 2024-03-18T15:34:28.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.188+0000 7f563789a000 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x563988d3e5e0 con 0x563988ef9dc0 2024-03-18T15:34:28.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.188+0000 7f56324b4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563988ef94a0 0x563988ef9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:28.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.188+0000 7f56324b4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563988ef94a0 0x563988ef9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45152/0 (socket says 172.21.15.5:45152) 2024-03-18T15:34:28.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.188+0000 7f56324b4640 1 -- 172.21.15.5:0/944276854 learned_addr learned my addr 172.21.15.5:0/944276854 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:28.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.188+0000 7f5631cb3640 1 --2- 172.21.15.5:0/944276854 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563988ef9dc0 0x563988f24180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:28.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.188+0000 7f56324b4640 1 -- 172.21.15.5:0/944276854 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563988ef9dc0 msgr2=0x563988f24180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:28.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.188+0000 7f56324b4640 1 --2- 172.21.15.5:0/944276854 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563988ef9dc0 0x563988f24180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:28.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.188+0000 7f56324b4640 1 -- 172.21.15.5:0/944276854 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563988f246c0 msgr2=0x563988f24aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:28.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.188+0000 7f56324b4640 1 --2- 172.21.15.5:0/944276854 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563988f246c0 0x563988f24aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:28.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.188+0000 7f56324b4640 1 -- 172.21.15.5:0/944276854 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563988cff6a0 con 0x563988ef94a0 2024-03-18T15:34:28.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.188+0000 7f56324b4640 1 --2- 172.21.15.5:0/944276854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563988ef94a0 0x563988ef9880 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5628083350 tx=0x7f562800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73cdd25e4c26e9d9 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:28.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.188+0000 7f560ffff640 1 -- 172.21.15.5:0/944276854 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562806d070 con 0x563988ef94a0 2024-03-18T15:34:28.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.188+0000 7f560ffff640 1 -- 172.21.15.5:0/944276854 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56280536a0 con 0x563988ef94a0 2024-03-18T15:34:28.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.188+0000 7f560ffff640 1 -- 172.21.15.5:0/944276854 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5628059160 con 0x563988ef94a0 2024-03-18T15:34:28.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.188+0000 7f563789a000 1 -- 172.21.15.5:0/944276854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563988ef94a0 msgr2=0x563988ef9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:28.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.188+0000 7f563789a000 1 --2- 172.21.15.5:0/944276854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563988ef94a0 0x563988ef9880 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5628083350 tx=0x7f562800cf90 comp rx=0 tx=0).stop 2024-03-18T15:34:28.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.188+0000 7f563789a000 1 -- 172.21.15.5:0/944276854 shutdown_connections 2024-03-18T15:34:28.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.188+0000 7f563789a000 1 --2- 172.21.15.5:0/944276854 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563988f246c0 0x563988f24aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:28.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.188+0000 7f563789a000 1 --2- 172.21.15.5:0/944276854 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563988ef9dc0 0x563988f24180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:28.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.188+0000 7f563789a000 1 --2- 172.21.15.5:0/944276854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563988ef94a0 0x563988ef9880 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:28.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.188+0000 7f563789a000 1 -- 172.21.15.5:0/944276854 >> 172.21.15.5:0/944276854 conn(0x563988f1ff30 msgr2=0x563988f22320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:28.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.188+0000 7f563789a000 1 -- 172.21.15.5:0/944276854 shutdown_connections 2024-03-18T15:34:28.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.188+0000 7f563789a000 1 -- 172.21.15.5:0/944276854 wait complete. 2024-03-18T15:34:28.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.188+0000 7f563789a000 1 -- start start 2024-03-18T15:34:28.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.188+0000 7f563789a000 1 Processor -- start 2024-03-18T15:34:28.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f563789a000 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563988ef94a0 0x563988ef2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:28.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f563789a000 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563988ef9dc0 0x563988ef3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:28.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f563789a000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563988f246c0 0x563988ef38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:28.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f563789a000 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563988d46b70 con 0x563988ef94a0 2024-03-18T15:34:28.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f563789a000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x563988d00850 con 0x563988f246c0 2024-03-18T15:34:28.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f563789a000 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x563988f26020 con 0x563988ef9dc0 2024-03-18T15:34:28.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f563789a000 10 client.0 _ll_register_callbacks cb 0x563988edf220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:34:28.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f56324b4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563988ef94a0 0x563988ef2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:28.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f56324b4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563988ef94a0 0x563988ef2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45158/0 (socket says 172.21.15.5:45158) 2024-03-18T15:34:28.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f56324b4640 1 -- 172.21.15.5:0/416607688 learned_addr learned my addr 172.21.15.5:0/416607688 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:28.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f5631cb3640 1 --2- 172.21.15.5:0/416607688 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563988ef9dc0 0x563988ef3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:28.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f56324b4640 1 -- 172.21.15.5:0/416607688 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563988ef9dc0 msgr2=0x563988ef3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:28.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f56324b4640 1 --2- 172.21.15.5:0/416607688 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563988ef9dc0 0x563988ef3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:28.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f56324b4640 1 -- 172.21.15.5:0/416607688 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563988f246c0 msgr2=0x563988ef38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:28.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f56324b4640 1 --2- 172.21.15.5:0/416607688 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563988f246c0 0x563988ef38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:28.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f56324b4640 1 -- 172.21.15.5:0/416607688 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563988cfd900 con 0x563988ef94a0 2024-03-18T15:34:28.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f5631cb3640 1 --2- 172.21.15.5:0/416607688 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563988ef9dc0 0x563988ef3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:28.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f56324b4640 1 --2- 172.21.15.5:0/416607688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563988ef94a0 0x563988ef2e50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5628007fd0 tx=0x7f5628000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37fea5dafe7bbf0a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:28.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f560d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-03-18T15:34:28.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f560d7fa640 1 -- 172.21.15.5:0/416607688 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5628072060 con 0x563988ef94a0 2024-03-18T15:34:28.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f563789a000 1 -- 172.21.15.5:0/416607688 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56280536a0 con 0x563988ef94a0 2024-03-18T15:34:28.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f560d7fa640 1 -- 172.21.15.5:0/416607688 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7f56280738b0 con 0x563988ef94a0 2024-03-18T15:34:28.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f560d7fa640 1 -- 172.21.15.5:0/416607688 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f5628078050 con 0x563988ef94a0 2024-03-18T15:34:28.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f563789a000 10 client.5145 fetch_fsmap learned FSMap version 16 2024-03-18T15:34:28.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f563789a000 1 -- 172.21.15.5:0/416607688 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x563988dc3940 con 0x563988ef94a0 2024-03-18T15:34:28.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f560d7fa640 1 -- 172.21.15.5:0/416607688 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56280536a0 con 0x563988ef94a0 2024-03-18T15:34:28.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f560d7fa640 1 -- 172.21.15.5:0/416607688 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 16) ==== 2284+0+0 (secure 0 0 0) 0x7f562807b020 con 0x563988ef94a0 2024-03-18T15:34:28.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f563789a000 10 client.5145 fetch_fsmap finished waiting for FSMap version 16 2024-03-18T15:34:28.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f563789a000 10 client.5145 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-18T15:34:28.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f563789a000 20 client.5145 populate_metadata read hostname 'smithi005' 2024-03-18T15:34:28.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f563789a000 1 --2- 172.21.15.5:0/416607688 >> [v2:172.21.15.5:6834/2235154713,v1:172.21.15.5:6835/2235154713] conn(0x563988f37560 0x563988f39950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:28.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f563789a000 4 client.5145 mds_command: new command op to 4968 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-18T15:34:28.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f563789a000 1 -- 172.21.15.5:0/416607688 --> [v2:172.21.15.5:6834/2235154713,v1:172.21.15.5:6835/2235154713] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563988cddbc0 con 0x563988f37560 2024-03-18T15:34:28.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f5631cb3640 1 --2- 172.21.15.5:0/416607688 >> [v2:172.21.15.5:6834/2235154713,v1:172.21.15.5:6835/2235154713] conn(0x563988f37560 0x563988f39950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:28.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f5631cb3640 1 --2- 172.21.15.5:0/416607688 >> [v2:172.21.15.5:6834/2235154713,v1:172.21.15.5:6835/2235154713] conn(0x563988f37560 0x563988f39950 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=325c37ebbf482f2 server_cookie=cfe6a6af9ffb62f9 in_seq=0 out_seq=0 2024-03-18T15:34:28.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f560d7fa640 10 client.5145 ms_handle_connect on v2:172.21.15.5:6834/2235154713 2024-03-18T15:34:28.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f560d7fa640 1 -- 172.21.15.5:0/416607688 <== mds.0 v2:172.21.15.5:6834/2235154713 1 ==== command_reply(tid 0: 0 ) ==== 8+0+22966 (crc 0 0 0) 0x563988cddbc0 con 0x563988f37560 2024-03-18T15:34:28.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f560d7fa640 10 client.5145 handle_command_reply: tid=0 2024-03-18T15:34:28.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f563789a000 1 client.5145 shutdown 2024-03-18T15:34:28.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f563789a000 1 -- 172.21.15.5:0/416607688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563988ef94a0 msgr2=0x563988ef2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:28.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f563789a000 1 --2- 172.21.15.5:0/416607688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563988ef94a0 0x563988ef2e50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5628007fd0 tx=0x7f5628000bd0 comp rx=0 tx=0).stop 2024-03-18T15:34:28.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f563789a000 1 -- 172.21.15.5:0/416607688 shutdown_connections 2024-03-18T15:34:28.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f563789a000 1 --2- 172.21.15.5:0/416607688 >> [v2:172.21.15.5:6834/2235154713,v1:172.21.15.5:6835/2235154713] conn(0x563988f37560 0x563988f39950 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:28.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f563789a000 1 --2- 172.21.15.5:0/416607688 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563988f246c0 0x563988ef38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:28.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f563789a000 1 --2- 172.21.15.5:0/416607688 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563988ef9dc0 0x563988ef3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:28.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f563789a000 1 --2- 172.21.15.5:0/416607688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563988ef94a0 0x563988ef2e50 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:28.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f563789a000 1 -- 172.21.15.5:0/416607688 >> 172.21.15.5:0/416607688 conn(0x563988f1ff30 msgr2=0x563988f22320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:28.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f560d7fa640 0 client.5145 ms_handle_reset on v2:172.21.15.5:6834/2235154713 2024-03-18T15:34:28.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f563789a000 1 -- 172.21.15.5:0/416607688 shutdown_connections 2024-03-18T15:34:28.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f563789a000 1 -- 172.21.15.5:0/416607688 wait complete. 2024-03-18T15:34:28.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.192+0000 7f563789a000 20 client.5145 trim_cache size 0 max 16384 2024-03-18T15:34:28.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.220+0000 7f563789a000 1 Processor -- start 2024-03-18T15:34:28.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.220+0000 7f563789a000 1 -- start start 2024-03-18T15:34:28.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.220+0000 7f563789a000 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563988ef94a0 0x563988f3d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:28.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.220+0000 7f563789a000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563988ef9dc0 0x563988f3a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:28.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.220+0000 7f563789a000 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563988f246c0 0x563988f3aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:28.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.220+0000 7f563789a000 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f562c1ec650 con 0x563988ef94a0 2024-03-18T15:34:28.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.220+0000 7f563789a000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f562c1eb880 con 0x563988ef9dc0 2024-03-18T15:34:28.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.220+0000 7f563789a000 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x563988dc34b0 con 0x563988f246c0 2024-03-18T15:34:28.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.220+0000 7f56324b4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563988ef94a0 0x563988f3d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:28.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.220+0000 7f56324b4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563988ef94a0 0x563988f3d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45162/0 (socket says 172.21.15.5:45162) 2024-03-18T15:34:28.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.220+0000 7f56324b4640 1 -- 172.21.15.5:0/1206219228 learned_addr learned my addr 172.21.15.5:0/1206219228 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:28.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.220+0000 7f56324b4640 1 -- 172.21.15.5:0/1206219228 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563988f246c0 msgr2=0x563988f3aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:28.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.220+0000 7f56324b4640 1 --2- 172.21.15.5:0/1206219228 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563988f246c0 0x563988f3aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:28.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.220+0000 7f56324b4640 1 -- 172.21.15.5:0/1206219228 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563988ef9dc0 msgr2=0x563988f3a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:28.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.220+0000 7f56324b4640 1 --2- 172.21.15.5:0/1206219228 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563988ef9dc0 0x563988f3a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:28.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.220+0000 7f56324b4640 1 -- 172.21.15.5:0/1206219228 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563988d9a870 con 0x563988ef94a0 2024-03-18T15:34:28.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.220+0000 7f56324b4640 1 --2- 172.21.15.5:0/1206219228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563988ef94a0 0x563988f3d220 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f56280050a0 tx=0x7f5628080000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17999fcc6f2e3079 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:28.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.220+0000 7f560f7fe640 1 -- 172.21.15.5:0/1206219228 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562806d050 con 0x563988ef94a0 2024-03-18T15:34:28.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.220+0000 7f560f7fe640 1 -- 172.21.15.5:0/1206219228 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56280536a0 con 0x563988ef94a0 2024-03-18T15:34:28.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.220+0000 7f560f7fe640 1 -- 172.21.15.5:0/1206219228 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56280547f0 con 0x563988ef94a0 2024-03-18T15:34:28.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.220+0000 7f563789a000 1 -- 172.21.15.5:0/1206219228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563988ef94a0 msgr2=0x563988f3d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:28.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.220+0000 7f563789a000 1 --2- 172.21.15.5:0/1206219228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563988ef94a0 0x563988f3d220 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f56280050a0 tx=0x7f5628080000 comp rx=0 tx=0).stop 2024-03-18T15:34:28.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.220+0000 7f563789a000 1 -- 172.21.15.5:0/1206219228 shutdown_connections 2024-03-18T15:34:28.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.220+0000 7f563789a000 1 --2- 172.21.15.5:0/1206219228 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563988f246c0 0x563988f3aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:28.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.220+0000 7f563789a000 1 --2- 172.21.15.5:0/1206219228 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563988ef9dc0 0x563988f3a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:28.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.220+0000 7f563789a000 1 --2- 172.21.15.5:0/1206219228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563988ef94a0 0x563988f3d220 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:28.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.220+0000 7f563789a000 1 -- 172.21.15.5:0/1206219228 >> 172.21.15.5:0/1206219228 conn(0x563988f1ff30 msgr2=0x563988f22bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:28.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.220+0000 7f563789a000 1 -- 172.21.15.5:0/1206219228 shutdown_connections 2024-03-18T15:34:28.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.220+0000 7f563789a000 1 -- 172.21.15.5:0/1206219228 wait complete. 2024-03-18T15:34:28.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.220+0000 7f563789a000 1 -- start start 2024-03-18T15:34:28.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.220+0000 7f563789a000 1 Processor -- start 2024-03-18T15:34:28.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f563789a000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563988ef94a0 0x563988f92630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:28.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f563789a000 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563988ef9dc0 0x563988f930a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:28.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f563789a000 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563988f246c0 0x563988f935e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:28.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f563789a000 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563988cda7b0 con 0x563988f246c0 2024-03-18T15:34:28.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f563789a000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x563988cfcf50 con 0x563988ef94a0 2024-03-18T15:34:28.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f563789a000 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x563988d41850 con 0x563988ef9dc0 2024-03-18T15:34:28.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f563789a000 10 client.0 _ll_register_callbacks cb 0x7f5628055aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-18T15:34:28.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f5632cb5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563988f246c0 0x563988f935e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:28.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f5632cb5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563988f246c0 0x563988f935e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45170/0 (socket says 172.21.15.5:45170) 2024-03-18T15:34:28.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f5632cb5640 1 -- 172.21.15.5:0/1406986031 learned_addr learned my addr 172.21.15.5:0/1406986031 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:28.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f5631cb3640 1 --2- 172.21.15.5:0/1406986031 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563988ef9dc0 0x563988f930a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:28.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f56324b4640 1 --2- 172.21.15.5:0/1406986031 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563988ef94a0 0x563988f92630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:28.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f5632cb5640 1 -- 172.21.15.5:0/1406986031 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563988ef9dc0 msgr2=0x563988f930a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:28.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f5632cb5640 1 --2- 172.21.15.5:0/1406986031 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563988ef9dc0 0x563988f930a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:28.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f5632cb5640 1 -- 172.21.15.5:0/1406986031 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563988ef94a0 msgr2=0x563988f92630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:28.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f5632cb5640 1 --2- 172.21.15.5:0/1406986031 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563988ef94a0 0x563988f92630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:28.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f5632cb5640 1 -- 172.21.15.5:0/1406986031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563988cf2070 con 0x563988f246c0 2024-03-18T15:34:28.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f5632cb5640 1 --2- 172.21.15.5:0/1406986031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563988f246c0 0x563988f935e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f562401afa0 tx=0x7f56240070f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5927fbb7bb5f4d3d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:28.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f560d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-03-18T15:34:28.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f560d7fa640 1 -- 172.21.15.5:0/1406986031 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56240bc070 con 0x563988f246c0 2024-03-18T15:34:28.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f560d7fa640 1 -- 172.21.15.5:0/1406986031 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7f56240a57b0 con 0x563988f246c0 2024-03-18T15:34:28.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f563789a000 1 -- 172.21.15.5:0/1406986031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56280536a0 con 0x563988f246c0 2024-03-18T15:34:28.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f560d7fa640 1 -- 172.21.15.5:0/1406986031 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f56240c7050 con 0x563988f246c0 2024-03-18T15:34:28.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f563789a000 10 client.5151 fetch_fsmap learned FSMap version 16 2024-03-18T15:34:28.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f563789a000 1 -- 172.21.15.5:0/1406986031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x563988cd6370 con 0x563988f246c0 2024-03-18T15:34:28.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f560d7fa640 1 -- 172.21.15.5:0/1406986031 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56280536a0 con 0x563988f246c0 2024-03-18T15:34:28.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f560d7fa640 1 -- 172.21.15.5:0/1406986031 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 16) ==== 2284+0+0 (secure 0 0 0) 0x7f56240ca020 con 0x563988f246c0 2024-03-18T15:34:28.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f563789a000 10 client.5151 fetch_fsmap finished waiting for FSMap version 16 2024-03-18T15:34:28.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f563789a000 10 client.5151 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-18T15:34:28.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f563789a000 20 client.5151 populate_metadata read hostname 'smithi005' 2024-03-18T15:34:28.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f563789a000 1 --2- 172.21.15.5:0/1406986031 >> [v2:172.21.15.5:6834/2235154713,v1:172.21.15.5:6835/2235154713] conn(0x563988f93b20 0x563988f93f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:28.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f563789a000 4 client.5151 mds_command: new command op to 4968 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-18T15:34:28.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f563789a000 1 -- 172.21.15.5:0/1406986031 --> [v2:172.21.15.5:6834/2235154713,v1:172.21.15.5:6835/2235154713] -- command(tid 0: {"prefix": "status"}) -- 0x563988d4fa80 con 0x563988f93b20 2024-03-18T15:34:28.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f56324b4640 1 --2- 172.21.15.5:0/1406986031 >> [v2:172.21.15.5:6834/2235154713,v1:172.21.15.5:6835/2235154713] conn(0x563988f93b20 0x563988f93f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:28.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f56324b4640 1 --2- 172.21.15.5:0/1406986031 >> [v2:172.21.15.5:6834/2235154713,v1:172.21.15.5:6835/2235154713] conn(0x563988f93b20 0x563988f93f00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5f646187f128a9a server_cookie=7d9b7810dd633100 in_seq=0 out_seq=0 2024-03-18T15:34:28.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f560d7fa640 10 client.5151 ms_handle_connect on v2:172.21.15.5:6834/2235154713 2024-03-18T15:34:28.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f560d7fa640 1 -- 172.21.15.5:0/1406986031 <== mds.0 v2:172.21.15.5:6834/2235154713 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x563988d4fa80 con 0x563988f93b20 2024-03-18T15:34:28.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f560d7fa640 10 client.5151 handle_command_reply: tid=0 2024-03-18T15:34:28.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f563789a000 1 client.5151 shutdown 2024-03-18T15:34:28.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f563789a000 1 -- 172.21.15.5:0/1406986031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563988f246c0 msgr2=0x563988f935e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:28.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f563789a000 1 --2- 172.21.15.5:0/1406986031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563988f246c0 0x563988f935e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f562401afa0 tx=0x7f56240070f0 comp rx=0 tx=0).stop 2024-03-18T15:34:28.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f563789a000 1 -- 172.21.15.5:0/1406986031 shutdown_connections 2024-03-18T15:34:28.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f563789a000 1 --2- 172.21.15.5:0/1406986031 >> [v2:172.21.15.5:6834/2235154713,v1:172.21.15.5:6835/2235154713] conn(0x563988f93b20 0x563988f93f00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:28.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f563789a000 1 --2- 172.21.15.5:0/1406986031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563988f246c0 0x563988f935e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:28.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f563789a000 1 --2- 172.21.15.5:0/1406986031 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x563988ef9dc0 0x563988f930a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:28.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f560d7fa640 0 client.5151 ms_handle_reset on v2:172.21.15.5:6834/2235154713 2024-03-18T15:34:28.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f563789a000 1 --2- 172.21.15.5:0/1406986031 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x563988ef94a0 0x563988f92630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:28.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f563789a000 1 -- 172.21.15.5:0/1406986031 >> 172.21.15.5:0/1406986031 conn(0x563988f1ff30 msgr2=0x563988f39570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:28.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f563789a000 1 -- 172.21.15.5:0/1406986031 shutdown_connections 2024-03-18T15:34:28.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f563789a000 1 -- 172.21.15.5:0/1406986031 wait complete. 2024-03-18T15:34:28.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f563789a000 20 client.5151 trim_cache size 0 max 16384 2024-03-18T15:34:28.242 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-03-18T15:34:28.242 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "ef54fe03-5aa7-4eb2-9b1e-66c9cc345b94", 2024-03-18T15:34:28.243 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-03-18T15:34:28.243 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 4968, 2024-03-18T15:34:28.243 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-03-18T15:34:28.243 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-03-18T15:34:28.243 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-03-18T15:34:28.243 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 5.4904561459999996, 2024-03-18T15:34:28.243 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 16, 2024-03-18T15:34:28.244 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 30, 2024-03-18T15:34:28.244 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 30, 2024-03-18T15:34:28.244 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 12.385300102 2024-03-18T15:34:28.244 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-03-18T15:34:28.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f5634718640 1 -- 172.21.15.5:0/3754524275 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5610067660 msgr2=0x7f5610069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:28.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f5634718640 1 --2- 172.21.15.5:0/3754524275 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5610067660 0x7f5610069b20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f562c068c20 tx=0x7f5620008040 comp rx=0 tx=0).stop 2024-03-18T15:34:28.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f5634718640 1 -- 172.21.15.5:0/3754524275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f562c0b5710 msgr2=0x7f562c1361a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:28.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f5634718640 1 --2- 172.21.15.5:0/3754524275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f562c0b5710 0x7f562c1361a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f562800c420 tx=0x7f562800c8e0 comp rx=0 tx=0).stop 2024-03-18T15:34:28.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f5634718640 1 -- 172.21.15.5:0/3754524275 shutdown_connections 2024-03-18T15:34:28.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f5634718640 1 --2- 172.21.15.5:0/3754524275 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5610067660 0x7f5610069b20 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:28.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f5634718640 1 --2- 172.21.15.5:0/3754524275 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f562c14b030 0x7f562c1e0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:28.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f5634718640 1 --2- 172.21.15.5:0/3754524275 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f562c141430 0x7f562c1dff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:28.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f5634718640 1 --2- 172.21.15.5:0/3754524275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f562c0b5710 0x7f562c1361a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:28.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f5634718640 1 -- 172.21.15.5:0/3754524275 >> 172.21.15.5:0/3754524275 conn(0x7f562c0673c0 msgr2=0x7f562c0b7de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:28.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f5634718640 1 -- 172.21.15.5:0/3754524275 shutdown_connections 2024-03-18T15:34:28.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.224+0000 7f5634718640 1 -- 172.21.15.5:0/3754524275 wait complete. 2024-03-18T15:34:28.248 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-18T15:34:28.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.356+0000 7f0bd355d640 1 Processor -- start 2024-03-18T15:34:28.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.356+0000 7f0bd355d640 1 -- start start 2024-03-18T15:34:28.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.356+0000 7f0bd355d640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0bcc06e390 0x7f0bcc070360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:28.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.356+0000 7f0bd355d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0bcc141260 0x7f0bcc1416c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:28.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.356+0000 7f0bd355d640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bcc06e770 con 0x7f0bcc147a20 2024-03-18T15:34:28.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.356+0000 7f0bd355d640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f0bcc072760 con 0x7f0bcc141260 2024-03-18T15:34:28.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.356+0000 7f0bd355d640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f0bcc072a40 con 0x7f0bcc06e390 2024-03-18T15:34:28.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.356+0000 7f0bd0af8640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0bcc141260 0x7f0bcc1416c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:28.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.356+0000 7f0bd12f9640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0bcc06e390 0x7f0bcc070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:28.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.356+0000 7f0bd1afa640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f0bcc147a20 0x7f0bcc147e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.5:43170/0 (socket says 172.21.15.5:43170) 2024-03-18T15:34:28.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.356+0000 7f0bd1afa640 1 -- 172.21.15.5:0/1265488686 learned_addr learned my addr 172.21.15.5:0/1265488686 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:28.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.356+0000 7f0bd0af8640 1 -- 172.21.15.5:0/1265488686 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0bcc06e390 msgr2=0x7f0bcc070360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:28.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.356+0000 7f0bd0af8640 1 --2- 172.21.15.5:0/1265488686 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0bcc06e390 0x7f0bcc070360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:28.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.356+0000 7f0bd0af8640 1 -- 172.21.15.5:0/1265488686 >> v1:172.21.15.80:6789/0 conn(0x7f0bcc147a20 legacy=0x7f0bcc147e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:28.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.356+0000 7f0bd0af8640 1 -- 172.21.15.5:0/1265488686 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bcc0713e0 con 0x7f0bcc141260 2024-03-18T15:34:28.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.360+0000 7f0bd0af8640 1 --2- 172.21.15.5:0/1265488686 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0bcc141260 0x7f0bcc1416c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc0097c0 tx=0x7f0bbc01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8b7a1f11cf69173 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:28.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.360+0000 7f0bc3fff640 1 -- 172.21.15.5:0/1265488686 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc0b0070 con 0x7f0bcc141260 2024-03-18T15:34:28.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.360+0000 7f0bc3fff640 1 -- 172.21.15.5:0/1265488686 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bbc0ab600 con 0x7f0bcc141260 2024-03-18T15:34:28.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.360+0000 7f0bc3fff640 1 -- 172.21.15.5:0/1265488686 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc0ab9d0 con 0x7f0bcc141260 2024-03-18T15:34:28.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.360+0000 7f0bd355d640 1 -- 172.21.15.5:0/1265488686 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0bcc141260 msgr2=0x7f0bcc1416c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:28.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.360+0000 7f0bd355d640 1 --2- 172.21.15.5:0/1265488686 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0bcc141260 0x7f0bcc1416c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc0097c0 tx=0x7f0bbc01c410 comp rx=0 tx=0).stop 2024-03-18T15:34:28.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.360+0000 7f0bd355d640 1 -- 172.21.15.5:0/1265488686 shutdown_connections 2024-03-18T15:34:28.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.360+0000 7f0bd355d640 1 --2- 172.21.15.5:0/1265488686 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0bcc141260 0x7f0bcc1416c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:28.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.360+0000 7f0bd355d640 1 --2- 172.21.15.5:0/1265488686 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0bcc06e390 0x7f0bcc070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:28.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.360+0000 7f0bd355d640 1 -- 172.21.15.5:0/1265488686 >> 172.21.15.5:0/1265488686 conn(0x7f0bcc0b6680 msgr2=0x7f0bcc0b6a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:28.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.360+0000 7f0bd355d640 1 -- 172.21.15.5:0/1265488686 shutdown_connections 2024-03-18T15:34:28.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.360+0000 7f0bd355d640 1 -- 172.21.15.5:0/1265488686 wait complete. 2024-03-18T15:34:28.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.360+0000 7f0bd355d640 1 Processor -- start 2024-03-18T15:34:28.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.360+0000 7f0bd355d640 1 -- start start 2024-03-18T15:34:28.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.360+0000 7f0bd355d640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0bcc06e390 0x7f0bcc1370d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:28.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.360+0000 7f0bd355d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bcc141260 0x7f0bcc13d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:28.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.360+0000 7f0bd355d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0bcc147a20 0x7f0bcc13d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:28.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.360+0000 7f0bd355d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc151580 con 0x7f0bcc141260 2024-03-18T15:34:28.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.360+0000 7f0bd355d640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0bcc151400 con 0x7f0bcc147a20 2024-03-18T15:34:28.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.360+0000 7f0bd355d640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f0bcc151700 con 0x7f0bcc06e390 2024-03-18T15:34:28.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.360+0000 7f0bd0af8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bcc141260 0x7f0bcc13d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:28.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.360+0000 7f0bd0af8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bcc141260 0x7f0bcc13d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45178/0 (socket says 172.21.15.5:45178) 2024-03-18T15:34:28.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.360+0000 7f0bd0af8640 1 -- 172.21.15.5:0/2190374573 learned_addr learned my addr 172.21.15.5:0/2190374573 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:28.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.360+0000 7f0bd12f9640 1 --2- 172.21.15.5:0/2190374573 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0bcc06e390 0x7f0bcc1370d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:28.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.360+0000 7f0bd1afa640 1 --2- 172.21.15.5:0/2190374573 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0bcc147a20 0x7f0bcc13d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:28.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.360+0000 7f0bd0af8640 1 -- 172.21.15.5:0/2190374573 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0bcc06e390 msgr2=0x7f0bcc1370d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:28.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.360+0000 7f0bd0af8640 1 --2- 172.21.15.5:0/2190374573 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0bcc06e390 0x7f0bcc1370d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:28.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.360+0000 7f0bd0af8640 1 -- 172.21.15.5:0/2190374573 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0bcc147a20 msgr2=0x7f0bcc13d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:28.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.360+0000 7f0bd0af8640 1 --2- 172.21.15.5:0/2190374573 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0bcc147a20 0x7f0bcc13d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:28.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.360+0000 7f0bd0af8640 1 -- 172.21.15.5:0/2190374573 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bcc13ca30 con 0x7f0bcc141260 2024-03-18T15:34:28.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.360+0000 7f0bd0af8640 1 --2- 172.21.15.5:0/2190374573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bcc141260 0x7f0bcc13d270 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc0ae040 tx=0x7f0bbc0191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:28.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.360+0000 7f0bc1ffb640 1 -- 172.21.15.5:0/2190374573 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc0b0040 con 0x7f0bcc141260 2024-03-18T15:34:28.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.360+0000 7f0bd355d640 1 -- 172.21.15.5:0/2190374573 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bcc1379c0 con 0x7f0bcc141260 2024-03-18T15:34:28.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.360+0000 7f0bc1ffb640 1 -- 172.21.15.5:0/2190374573 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bbc0b9070 con 0x7f0bcc141260 2024-03-18T15:34:28.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.360+0000 7f0bc1ffb640 1 -- 172.21.15.5:0/2190374573 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bbc006e20 con 0x7f0bcc141260 2024-03-18T15:34:28.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.364+0000 7f0bd355d640 1 -- 172.21.15.5:0/2190374573 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bcc13e3d0 con 0x7f0bcc141260 2024-03-18T15:34:28.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.364+0000 7f0bd355d640 1 -- 172.21.15.5:0/2190374573 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bcc04f550 con 0x7f0bcc141260 2024-03-18T15:34:28.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.364+0000 7f0bc1ffb640 1 -- 172.21.15.5:0/2190374573 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f0bbc0b4650 con 0x7f0bcc141260 2024-03-18T15:34:28.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.364+0000 7f0bc1ffb640 1 --2- 172.21.15.5:0/2190374573 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0bb0067660 0x7f0bb0069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:28.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.364+0000 7f0bd12f9640 1 --2- 172.21.15.5:0/2190374573 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0bb0067660 0x7f0bb0069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:28.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.364+0000 7f0bc1ffb640 1 -- 172.21.15.5:0/2190374573 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7f0bbc11f620 con 0x7f0bcc141260 2024-03-18T15:34:28.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.364+0000 7f0bd12f9640 1 --2- 172.21.15.5:0/2190374573 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0bb0067660 0x7f0bb0069b20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8009a50 tx=0x7f0bc8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:28.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.372+0000 7f0bc1ffb640 1 -- 172.21.15.5:0/2190374573 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f0bbc016090 con 0x7f0bcc141260 2024-03-18T15:34:28.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.564+0000 7f0bd355d640 1 -- 172.21.15.5:0/2190374573 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0bcc06b420 con 0x7f0bcc141260 2024-03-18T15:34:28.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.568+0000 7f0bc1ffb640 1 -- 172.21.15.5:0/2190374573 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+73 (secure 0 0 0) 0x7f0bbc0f02b0 con 0x7f0bcc141260 2024-03-18T15:34:28.569 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-18T15:34:28.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.568+0000 7f0bd355d640 1 -- 172.21.15.5:0/2190374573 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0bb0067660 msgr2=0x7f0bb0069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:28.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.568+0000 7f0bd355d640 1 --2- 172.21.15.5:0/2190374573 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0bb0067660 0x7f0bb0069b20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8009a50 tx=0x7f0bc8008040 comp rx=0 tx=0).stop 2024-03-18T15:34:28.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.568+0000 7f0bd355d640 1 -- 172.21.15.5:0/2190374573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bcc141260 msgr2=0x7f0bcc13d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:28.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.568+0000 7f0bd355d640 1 --2- 172.21.15.5:0/2190374573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bcc141260 0x7f0bcc13d270 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc0ae040 tx=0x7f0bbc0191b0 comp rx=0 tx=0).stop 2024-03-18T15:34:28.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.568+0000 7f0bd355d640 1 -- 172.21.15.5:0/2190374573 shutdown_connections 2024-03-18T15:34:28.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.568+0000 7f0bd355d640 1 --2- 172.21.15.5:0/2190374573 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0bb0067660 0x7f0bb0069b20 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:28.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.568+0000 7f0bd355d640 1 --2- 172.21.15.5:0/2190374573 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0bcc147a20 0x7f0bcc13d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:28.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.568+0000 7f0bd355d640 1 --2- 172.21.15.5:0/2190374573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bcc141260 0x7f0bcc13d270 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:28.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.568+0000 7f0bd355d640 1 --2- 172.21.15.5:0/2190374573 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0bcc06e390 0x7f0bcc1370d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:28.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.568+0000 7f0bd355d640 1 -- 172.21.15.5:0/2190374573 >> 172.21.15.5:0/2190374573 conn(0x7f0bcc0b6680 msgr2=0x7f0bcc13f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:28.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.568+0000 7f0bd355d640 1 -- 172.21.15.5:0/2190374573 shutdown_connections 2024-03-18T15:34:28.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28.568+0000 7f0bd355d640 1 -- 172.21.15.5:0/2190374573 wait complete. 2024-03-18T15:34:28.594 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-18T15:34:28.594 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-18T15:34:28.594 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-03-18T15:34:28.594 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-18T15:34:28.594 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-18T15:34:28.595 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-18T15:34:28.595 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-18T15:34:28.595 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-18T15:34:28.595 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-18T15:34:28.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:34:28.595 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-03-18T15:34:28.600 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-18T15:34:28.600 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-18T15:34:28.600 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-03-18T15:34:28.600 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-03-18T15:34:28.600 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-03-18T15:34:28.600 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-03-18T15:34:28.600 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-18T15:34:28.600 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-03-18T15:34:28.600 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-03-18T15:34:28.600 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-18T15:34:28.601 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-03-18T15:34:28.601 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-03-18T15:34:28.601 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-18T15:34:28.601 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-03-18T15:34:28.601 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-03-18T15:34:28.601 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-03-18T15:34:28.601 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42429sec preferred_lft 42429sec 2024-03-18T15:34:28.601 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link 2024-03-18T15:34:28.601 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-03-18T15:34:28.601 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-18T15:34:28.601 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-03-18T15:34:28.601 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-03-18T15:34:28.601 INFO:teuthology.orchestra.run.smithi005.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-18T15:34:28.601 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 02:42:57:05:8c:91 brd ff:ff:ff:ff:ff:ff 2024-03-18T15:34:28.601 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-18T15:34:28.602 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-03-18T15:34:28.602 INFO:teuthology.orchestra.run.smithi005.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-18T15:34:28.602 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-18T15:34:28.602 INFO:teuthology.orchestra.run.smithi005.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-18T15:34:28.602 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-03-18T15:34:28.602 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-18T15:34:28.602 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-03-18T15:34:28.602 INFO:teuthology.orchestra.run.smithi005.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-18T15:34:28.602 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-18T15:34:28.602 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-18T15:34:28.602 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-03-18T15:34:28.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:34:28.603 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-18T15:34:28.651 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-03-18T15:34:28.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:34:28.653 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-03-18T15:34:28.702 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-03-18T15:34:28.702 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-18T15:34:28.702 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-03-18T15:34:28.703 INFO:teuthology.orchestra.run.smithi005.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-18T15:34:28.703 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-03-18T15:34:28.703 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-18T15:34:28.703 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-03-18T15:34:28.703 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-03-18T15:34:28.703 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-03-18T15:34:28.703 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-03-18T15:34:28.703 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-03-18T15:34:28.703 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-03-18T15:34:28.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-03-18T15:34:28.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T15:34:28.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:34:28.821 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-03-18T15:34:28.826 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-18T15:34:28.826 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-18T15:34:28.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:34:28.826 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-03-18T15:34:28.879 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-18T15:34:28.880 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-18T15:34:28.880 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-18T15:34:28.926 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-18T15:34:28.926 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-18T15:34:28.927 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-18T15:34:28.973 DEBUG:teuthology.orchestra.run.smithi005:> sudo modprobe fuse 2024-03-18T15:34:29.031 DEBUG:teuthology.orchestra.run.smithi005:> cat /proc/self/mounts | awk '{print $2}' 2024-03-18T15:34:29.078 INFO:teuthology.orchestra.run.smithi005.stdout:/sys 2024-03-18T15:34:29.079 INFO:teuthology.orchestra.run.smithi005.stdout:/proc 2024-03-18T15:34:29.079 INFO:teuthology.orchestra.run.smithi005.stdout:/dev 2024-03-18T15:34:29.079 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/pts 2024-03-18T15:34:29.079 INFO:teuthology.orchestra.run.smithi005.stdout:/run 2024-03-18T15:34:29.079 INFO:teuthology.orchestra.run.smithi005.stdout:/ 2024-03-18T15:34:29.079 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/security 2024-03-18T15:34:29.079 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/shm 2024-03-18T15:34:29.079 INFO:teuthology.orchestra.run.smithi005.stdout:/run/lock 2024-03-18T15:34:29.079 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup 2024-03-18T15:34:29.079 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/pstore 2024-03-18T15:34:29.079 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/bpf 2024-03-18T15:34:29.079 INFO:teuthology.orchestra.run.smithi005.stdout:/proc/sys/fs/binfmt_misc 2024-03-18T15:34:29.080 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/hugepages 2024-03-18T15:34:29.080 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/mqueue 2024-03-18T15:34:29.080 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/debug 2024-03-18T15:34:29.080 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/tracing 2024-03-18T15:34:29.080 INFO:teuthology.orchestra.run.smithi005.stdout:/proc/fs/nfsd 2024-03-18T15:34:29.080 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/fuse/connections 2024-03-18T15:34:29.080 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/config 2024-03-18T15:34:29.080 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-sysusers.service 2024-03-18T15:34:29.080 INFO:teuthology.orchestra.run.smithi005.stdout:/run/rpc_pipefs 2024-03-18T15:34:29.080 INFO:teuthology.orchestra.run.smithi005.stdout:/run/user/1000 2024-03-18T15:34:29.080 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph 2024-03-18T15:34:29.080 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-0 2024-03-18T15:34:29.080 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-1 2024-03-18T15:34:29.080 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-2 2024-03-18T15:34:29.080 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-3 2024-03-18T15:34:29.081 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns 2024-03-18T15:34:29.081 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-18T15:34:29.081 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-18T15:34:29.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:34:29.082 DEBUG:teuthology.orchestra.run.smithi005:> ls /sys/fs/fuse/connections 2024-03-18T15:34:29.129 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-18T15:34:29.129 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-18T15:34:29.175 DEBUG:teuthology.orchestra.run.smithi005:> sudo modprobe fuse 2024-03-18T15:34:29.186 DEBUG:teuthology.orchestra.run.smithi005:> cat /proc/self/mounts | awk '{print $2}' 2024-03-18T15:34:29.234 INFO:teuthology.orchestra.run.smithi005.stdout:/sys 2024-03-18T15:34:29.234 INFO:teuthology.orchestra.run.smithi005.stdout:/proc 2024-03-18T15:34:29.234 INFO:teuthology.orchestra.run.smithi005.stdout:/dev 2024-03-18T15:34:29.234 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/pts 2024-03-18T15:34:29.234 INFO:teuthology.orchestra.run.smithi005.stdout:/run 2024-03-18T15:34:29.234 INFO:teuthology.orchestra.run.smithi005.stdout:/ 2024-03-18T15:34:29.234 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/security 2024-03-18T15:34:29.234 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/shm 2024-03-18T15:34:29.235 INFO:teuthology.orchestra.run.smithi005.stdout:/run/lock 2024-03-18T15:34:29.235 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup 2024-03-18T15:34:29.235 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/pstore 2024-03-18T15:34:29.235 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/bpf 2024-03-18T15:34:29.235 INFO:teuthology.orchestra.run.smithi005.stdout:/proc/sys/fs/binfmt_misc 2024-03-18T15:34:29.235 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/hugepages 2024-03-18T15:34:29.235 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/mqueue 2024-03-18T15:34:29.235 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/debug 2024-03-18T15:34:29.235 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/tracing 2024-03-18T15:34:29.235 INFO:teuthology.orchestra.run.smithi005.stdout:/proc/fs/nfsd 2024-03-18T15:34:29.235 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/fuse/connections 2024-03-18T15:34:29.235 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/config 2024-03-18T15:34:29.235 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-sysusers.service 2024-03-18T15:34:29.235 INFO:teuthology.orchestra.run.smithi005.stdout:/run/rpc_pipefs 2024-03-18T15:34:29.236 INFO:teuthology.orchestra.run.smithi005.stdout:/run/user/1000 2024-03-18T15:34:29.236 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph 2024-03-18T15:34:29.236 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-0 2024-03-18T15:34:29.236 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-1 2024-03-18T15:34:29.236 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-2 2024-03-18T15:34:29.236 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-3 2024-03-18T15:34:29.236 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns 2024-03-18T15:34:29.236 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-18T15:34:29.236 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-18T15:34:29.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:34:29.237 DEBUG:teuthology.orchestra.run.smithi005:> ls /sys/fs/fuse/connections 2024-03-18T15:34:29.271 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi005.stderr:2024-03-18T15:34:29.268+0000 7f1b3278d580 -1 init, newargv = 0x55a5ee1aa5e0 newargc=15 2024-03-18T15:34:29.271 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi005.stderr:ceph-fuse[35096]: starting ceph client 2024-03-18T15:34:29.276 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi005.stderr:ceph-fuse[35096]: starting fuse 2024-03-18T15:34:30.377 DEBUG:teuthology.orchestra.run.smithi005:> sudo modprobe fuse 2024-03-18T15:34:30.411 DEBUG:teuthology.orchestra.run.smithi005:> cat /proc/self/mounts | awk '{print $2}' 2024-03-18T15:34:30.465 INFO:teuthology.orchestra.run.smithi005.stdout:/sys 2024-03-18T15:34:30.465 INFO:teuthology.orchestra.run.smithi005.stdout:/proc 2024-03-18T15:34:30.465 INFO:teuthology.orchestra.run.smithi005.stdout:/dev 2024-03-18T15:34:30.465 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/pts 2024-03-18T15:34:30.465 INFO:teuthology.orchestra.run.smithi005.stdout:/run 2024-03-18T15:34:30.466 INFO:teuthology.orchestra.run.smithi005.stdout:/ 2024-03-18T15:34:30.466 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/security 2024-03-18T15:34:30.466 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/shm 2024-03-18T15:34:30.466 INFO:teuthology.orchestra.run.smithi005.stdout:/run/lock 2024-03-18T15:34:30.466 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/cgroup 2024-03-18T15:34:30.466 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/pstore 2024-03-18T15:34:30.466 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/bpf 2024-03-18T15:34:30.466 INFO:teuthology.orchestra.run.smithi005.stdout:/proc/sys/fs/binfmt_misc 2024-03-18T15:34:30.466 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/hugepages 2024-03-18T15:34:30.466 INFO:teuthology.orchestra.run.smithi005.stdout:/dev/mqueue 2024-03-18T15:34:30.466 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/debug 2024-03-18T15:34:30.466 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/tracing 2024-03-18T15:34:30.466 INFO:teuthology.orchestra.run.smithi005.stdout:/proc/fs/nfsd 2024-03-18T15:34:30.467 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/fs/fuse/connections 2024-03-18T15:34:30.467 INFO:teuthology.orchestra.run.smithi005.stdout:/sys/kernel/config 2024-03-18T15:34:30.467 INFO:teuthology.orchestra.run.smithi005.stdout:/run/credentials/systemd-sysusers.service 2024-03-18T15:34:30.467 INFO:teuthology.orchestra.run.smithi005.stdout:/run/rpc_pipefs 2024-03-18T15:34:30.467 INFO:teuthology.orchestra.run.smithi005.stdout:/run/user/1000 2024-03-18T15:34:30.467 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph 2024-03-18T15:34:30.467 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-0 2024-03-18T15:34:30.467 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-1 2024-03-18T15:34:30.467 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-2 2024-03-18T15:34:30.467 INFO:teuthology.orchestra.run.smithi005.stdout:/var/lib/ceph/osd/ceph-3 2024-03-18T15:34:30.467 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns 2024-03-18T15:34:30.467 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-18T15:34:30.467 INFO:teuthology.orchestra.run.smithi005.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-18T15:34:30.467 INFO:teuthology.orchestra.run.smithi005.stdout:/home/ubuntu/cephtest/mnt.0 2024-03-18T15:34:30.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:34:30.468 DEBUG:teuthology.orchestra.run.smithi005:> ls /sys/fs/fuse/connections 2024-03-18T15:34:30.524 INFO:teuthology.orchestra.run.smithi005.stdout:45 2024-03-18T15:34:30.524 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-03-18T15:34:30.524 DEBUG:teuthology.orchestra.run.smithi005:> sudo stdin-killer -- python3 -c ' 2024-03-18T15:34:30.525 DEBUG:teuthology.orchestra.run.smithi005:> import glob 2024-03-18T15:34:30.525 DEBUG:teuthology.orchestra.run.smithi005:> import re 2024-03-18T15:34:30.525 DEBUG:teuthology.orchestra.run.smithi005:> import os 2024-03-18T15:34:30.525 DEBUG:teuthology.orchestra.run.smithi005:> import subprocess 2024-03-18T15:34:30.525 DEBUG:teuthology.orchestra.run.smithi005:> 2024-03-18T15:34:30.525 DEBUG:teuthology.orchestra.run.smithi005:> def _find_admin_socket(client_name): 2024-03-18T15:34:30.525 DEBUG:teuthology.orchestra.run.smithi005:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-18T15:34:30.525 DEBUG:teuthology.orchestra.run.smithi005:> files = glob.glob(asok_path) 2024-03-18T15:34:30.525 DEBUG:teuthology.orchestra.run.smithi005:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-18T15:34:30.525 DEBUG:teuthology.orchestra.run.smithi005:> 2024-03-18T15:34:30.525 DEBUG:teuthology.orchestra.run.smithi005:> # Given a non-glob path, it better be there 2024-03-18T15:34:30.525 DEBUG:teuthology.orchestra.run.smithi005:> if "*" not in asok_path: 2024-03-18T15:34:30.525 DEBUG:teuthology.orchestra.run.smithi005:> assert(len(files) == 1) 2024-03-18T15:34:30.525 DEBUG:teuthology.orchestra.run.smithi005:> return files[0] 2024-03-18T15:34:30.526 DEBUG:teuthology.orchestra.run.smithi005:> 2024-03-18T15:34:30.526 DEBUG:teuthology.orchestra.run.smithi005:> for f in files: 2024-03-18T15:34:30.526 DEBUG:teuthology.orchestra.run.smithi005:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-18T15:34:30.526 DEBUG:teuthology.orchestra.run.smithi005:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-18T15:34:30.526 DEBUG:teuthology.orchestra.run.smithi005:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-18T15:34:30.526 DEBUG:teuthology.orchestra.run.smithi005:> contents = proc_f.read() 2024-03-18T15:34:30.526 DEBUG:teuthology.orchestra.run.smithi005:> if mountpoint in contents: 2024-03-18T15:34:30.526 DEBUG:teuthology.orchestra.run.smithi005:> return f 2024-03-18T15:34:30.526 DEBUG:teuthology.orchestra.run.smithi005:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-18T15:34:30.526 DEBUG:teuthology.orchestra.run.smithi005:> 2024-03-18T15:34:30.526 DEBUG:teuthology.orchestra.run.smithi005:> print(_find_admin_socket("client.0")) 2024-03-18T15:34:30.526 DEBUG:teuthology.orchestra.run.smithi005:> ' 2024-03-18T15:34:30.649 INFO:teuthology.orchestra.run.smithi005.stdout:/var/run/ceph/ceph-client.0.35096.asok 2024-03-18T15:34:30.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T15:34:30.661 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.35096.asok 2024-03-18T15:34:30.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:34:30.662 DEBUG:teuthology.orchestra.run.smithi005:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.35096.asok status 2024-03-18T15:34:30.824 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-03-18T15:34:30.825 INFO:teuthology.orchestra.run.smithi005.stdout: "metadata": { 2024-03-18T15:34:30.825 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_sha1": "3d78f5ba07bf18880172c07d4d7b4ac00c9d30af", 2024-03-18T15:34:30.825 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_version": "ceph version 19.0.0-2162-g3d78f5ba (3d78f5ba07bf18880172c07d4d7b4ac00c9d30af) squid (dev)", 2024-03-18T15:34:30.825 INFO:teuthology.orchestra.run.smithi005.stdout: "entity_id": "0", 2024-03-18T15:34:30.825 INFO:teuthology.orchestra.run.smithi005.stdout: "hostname": "smithi005", 2024-03-18T15:34:30.825 INFO:teuthology.orchestra.run.smithi005.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-18T15:34:30.825 INFO:teuthology.orchestra.run.smithi005.stdout: "pid": "35096", 2024-03-18T15:34:30.825 INFO:teuthology.orchestra.run.smithi005.stdout: "root": "/" 2024-03-18T15:34:30.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-03-18T15:34:30.825 INFO:teuthology.orchestra.run.smithi005.stdout: "dentry_count": 0, 2024-03-18T15:34:30.825 INFO:teuthology.orchestra.run.smithi005.stdout: "dentry_pinned_count": 0, 2024-03-18T15:34:30.826 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 5160, 2024-03-18T15:34:30.826 INFO:teuthology.orchestra.run.smithi005.stdout: "inst": { 2024-03-18T15:34:30.826 INFO:teuthology.orchestra.run.smithi005.stdout: "name": { 2024-03-18T15:34:30.826 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "client", 2024-03-18T15:34:30.826 INFO:teuthology.orchestra.run.smithi005.stdout: "num": 5160 2024-03-18T15:34:30.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-03-18T15:34:30.826 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": { 2024-03-18T15:34:30.826 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-03-18T15:34:30.826 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "192.168.0.1:0", 2024-03-18T15:34:30.826 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2600719084 2024-03-18T15:34:30.826 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-03-18T15:34:30.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-03-18T15:34:30.826 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": { 2024-03-18T15:34:30.826 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-03-18T15:34:30.827 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "192.168.0.1:0", 2024-03-18T15:34:30.827 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2600719084 2024-03-18T15:34:30.827 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-03-18T15:34:30.827 INFO:teuthology.orchestra.run.smithi005.stdout: "inst_str": "client.5160 192.168.0.1:0/2600719084", 2024-03-18T15:34:30.827 INFO:teuthology.orchestra.run.smithi005.stdout: "addr_str": "192.168.0.1:0/2600719084", 2024-03-18T15:34:30.827 INFO:teuthology.orchestra.run.smithi005.stdout: "inode_count": 1, 2024-03-18T15:34:30.827 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_epoch": 16, 2024-03-18T15:34:30.827 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_epoch": 30, 2024-03-18T15:34:30.827 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_epoch_barrier": 0, 2024-03-18T15:34:30.827 INFO:teuthology.orchestra.run.smithi005.stdout: "blocklisted": false, 2024-03-18T15:34:30.827 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs" 2024-03-18T15:34:30.827 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-03-18T15:34:30.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:34:30.839 DEBUG:teuthology.orchestra.run.smithi005:> stat --file-system '--printf=%T 2024-03-18T15:34:30.839 DEBUG:teuthology.orchestra.run.smithi005:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-18T15:34:30.846 INFO:teuthology.orchestra.run.smithi005.stdout:fuseblk 2024-03-18T15:34:30.846 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-18T15:34:30.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:34:30.846 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-18T15:34:30.901 DEBUG:teuthology.orchestra.run.smithi005:> mktemp 2024-03-18T15:34:30.951 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/tmp.8ppnANCBcM 2024-03-18T15:34:30.951 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-03-18T15:34:30.951 DEBUG:teuthology.orchestra.run.smithi005:> dd of=/tmp/tmp.8ppnANCBcM 2024-03-18T15:34:31.002 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'mkdir -p dir1/dir2/dir3') 2024-03-18T15:34:31.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-18T15:34:31.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T15:34:31.130 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'chmod 755 /tmp/test-cephfs12wtjnq0') 2024-03-18T15:34:31.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-03-18T15:34:31.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-18T15:34:31.188 INFO:tasks.cephfs.test_cephfs_shell:Running script "/tmp/test-cephfs12wtjnq0" 2024-03-18T15:34:31.188 DEBUG:teuthology.orchestra.run.smithi005:> cephfs-shell -b /tmp/test-cephfs12wtjnq0 2024-03-18T15:34:31.515 INFO:teuthology.orchestra.run.smithi005.stderr:Unrecognized command 2024-03-18T15:34:31.545 DEBUG:teuthology.orchestra.run:got remote process result: 127 2024-03-18T15:34:31.546 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-18T15:34:31.546 DEBUG:teuthology.orchestra.run.smithi005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-18T15:34:31.594 DEBUG:teuthology.orchestra.run.smithi080:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-18T15:34:31.624 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-18T15:34:31.624 INFO:tasks.cephfs.mount:Terminating background process 2024-03-18T15:34:31.624 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-03-18T15:34:31.625 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-18T15:34:31.645 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:34:31.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.816+0000 7f3d857f6640 1 Processor -- start 2024-03-18T15:34:31.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.820+0000 7f3d857f6640 1 -- start start 2024-03-18T15:34:31.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.820+0000 7f3d857f6640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d8014b660 0x7f3d80070360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:31.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.820+0000 7f3d857f6640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d8006b8c0 con 0x7f3d801410d0 2024-03-18T15:34:31.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.820+0000 7f3d857f6640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d8014ab80 con 0x7f3d8006b4e0 2024-03-18T15:34:31.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.820+0000 7f3d857f6640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f3d8014b1d0 con 0x7f3d8014b660 2024-03-18T15:34:31.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.820+0000 7f3d7f7fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f3d801410d0 0x7f3d80141560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44264/0 (socket says 172.21.15.5:44264) 2024-03-18T15:34:31.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.820+0000 7f3d7f7fe640 1 -- 172.21.15.5:0/2780901984 learned_addr learned my addr 172.21.15.5:0/2780901984 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:31.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.820+0000 7f3d77fff640 1 --2- 172.21.15.5:0/2780901984 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d8014b660 0x7f3d80070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:31.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.820+0000 7f3d7e7fc640 1 -- 172.21.15.5:0/2780901984 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 323416923 0 0) 0x7f3d8006b8c0 con 0x7f3d801410d0 2024-03-18T15:34:31.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.820+0000 7f3d7e7fc640 1 -- 172.21.15.5:0/2780901984 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d6c003530 con 0x7f3d801410d0 2024-03-18T15:34:31.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.820+0000 7f3d7e7fc640 1 -- 172.21.15.5:0/2780901984 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 942769910 0 0) 0x7f3d6c003530 con 0x7f3d801410d0 2024-03-18T15:34:31.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.820+0000 7f3d7e7fc640 1 -- 172.21.15.5:0/2780901984 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d8014b660 msgr2=0x7f3d80070360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:31.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.820+0000 7f3d7e7fc640 1 --2- 172.21.15.5:0/2780901984 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d8014b660 0x7f3d80070360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:31.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.820+0000 7f3d7e7fc640 1 -- 172.21.15.5:0/2780901984 >> v1:172.21.15.80:6789/0 conn(0x7f3d8006b4e0 legacy=0x7f3d80071da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:31.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.820+0000 7f3d7e7fc640 1 -- 172.21.15.5:0/2780901984 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d8014ae60 con 0x7f3d801410d0 2024-03-18T15:34:31.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.820+0000 7f3d7e7fc640 1 -- 172.21.15.5:0/2780901984 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3837549003 0 0) 0x7f3d78001a00 con 0x7f3d801410d0 2024-03-18T15:34:31.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.820+0000 7f3d7e7fc640 1 -- 172.21.15.5:0/2780901984 >> v1:172.21.15.5:6789/0 conn(0x7f3d801410d0 legacy=0x7f3d80141560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:31.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.820+0000 7f3d7e7fc640 1 --2- 172.21.15.5:0/2780901984 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d8014b660 0x7f3d6c003ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:31.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.820+0000 7f3d7e7fc640 1 --2- 172.21.15.5:0/2780901984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d6c004050 0x7f3d6c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:31.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.820+0000 7f3d7e7fc640 1 --2- 172.21.15.5:0/2780901984 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d6c004d50 0x7f3d6c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:31.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.820+0000 7f3d7e7fc640 1 -- 172.21.15.5:0/2780901984 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8014b1d0 con 0x7f3d6c004050 2024-03-18T15:34:31.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.820+0000 7f3d7e7fc640 1 -- 172.21.15.5:0/2780901984 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6c00bc30 con 0x7f3d6c004d50 2024-03-18T15:34:31.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.820+0000 7f3d7e7fc640 1 -- 172.21.15.5:0/2780901984 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f3d6c00be30 con 0x7f3d8014b660 2024-03-18T15:34:31.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.820+0000 7f3d77fff640 1 --2- 172.21.15.5:0/2780901984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d6c004050 0x7f3d6c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:31.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.820+0000 7f3d7effd640 1 --2- 172.21.15.5:0/2780901984 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d8014b660 0x7f3d6c003ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:31.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.820+0000 7f3d7f7fe640 1 --2- 172.21.15.5:0/2780901984 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d6c004d50 0x7f3d6c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:31.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.820+0000 7f3d77fff640 1 -- 172.21.15.5:0/2780901984 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d8014b660 msgr2=0x7f3d6c003ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:31.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.820+0000 7f3d77fff640 1 --2- 172.21.15.5:0/2780901984 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d8014b660 0x7f3d6c003ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:31.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.820+0000 7f3d77fff640 1 -- 172.21.15.5:0/2780901984 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d6c004d50 msgr2=0x7f3d6c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:31.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.820+0000 7f3d77fff640 1 --2- 172.21.15.5:0/2780901984 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d6c004d50 0x7f3d6c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:31.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.820+0000 7f3d77fff640 1 -- 172.21.15.5:0/2780901984 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d6c003530 con 0x7f3d6c004050 2024-03-18T15:34:31.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.820+0000 7f3d7f7fe640 1 --2- 172.21.15.5:0/2780901984 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d6c004d50 0x7f3d6c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:31.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.824+0000 7f3d77fff640 1 --2- 172.21.15.5:0/2780901984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d6c004050 0x7f3d6c004810 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3d68009ae0 tx=0x7f3d6801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c6a7c89b482fcb1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:31.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.824+0000 7f3d7e7fc640 1 -- 172.21.15.5:0/2780901984 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d680096b0 con 0x7f3d6c004050 2024-03-18T15:34:31.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.824+0000 7f3d7e7fc640 1 -- 172.21.15.5:0/2780901984 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d680a6450 con 0x7f3d6c004050 2024-03-18T15:34:31.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.824+0000 7f3d7e7fc640 1 -- 172.21.15.5:0/2780901984 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d680a6840 con 0x7f3d6c004050 2024-03-18T15:34:31.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.824+0000 7f3d857f6640 1 -- 172.21.15.5:0/2780901984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d6c004050 msgr2=0x7f3d6c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:31.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.824+0000 7f3d857f6640 1 --2- 172.21.15.5:0/2780901984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d6c004050 0x7f3d6c004810 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3d68009ae0 tx=0x7f3d6801ca70 comp rx=0 tx=0).stop 2024-03-18T15:34:31.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.824+0000 7f3d7f7fe640 1 -- 172.21.15.5:0/2780901984 reap_dead start 2024-03-18T15:34:31.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.824+0000 7f3d857f6640 1 -- 172.21.15.5:0/2780901984 shutdown_connections 2024-03-18T15:34:31.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.824+0000 7f3d857f6640 1 -- 172.21.15.5:0/2780901984 >> 172.21.15.5:0/2780901984 conn(0x7f3d800b6680 msgr2=0x7f3d800b6a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:31.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.824+0000 7f3d857f6640 1 -- 172.21.15.5:0/2780901984 shutdown_connections 2024-03-18T15:34:31.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.824+0000 7f3d857f6640 1 -- 172.21.15.5:0/2780901984 wait complete. 2024-03-18T15:34:31.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.824+0000 7f3d857f6640 1 Processor -- start 2024-03-18T15:34:31.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.824+0000 7f3d857f6640 1 -- start start 2024-03-18T15:34:31.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.828+0000 7f3d857f6640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d6c004050 0x7f3d800b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:31.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.828+0000 7f3d857f6640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d80145e10 0x7f3d80146c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:31.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.828+0000 7f3d857f6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d80150f80 0x7f3d800ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:31.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.828+0000 7f3d857f6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3d801537a0 con 0x7f3d80150f80 2024-03-18T15:34:31.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.828+0000 7f3d857f6640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3d80071630 con 0x7f3d80145e10 2024-03-18T15:34:31.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.828+0000 7f3d857f6640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f3d80153920 con 0x7f3d6c004050 2024-03-18T15:34:31.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.828+0000 7f3d7f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d80150f80 0x7f3d800ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:31.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.828+0000 7f3d7f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d80150f80 0x7f3d800ad8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56378/0 (socket says 172.21.15.5:56378) 2024-03-18T15:34:31.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.828+0000 7f3d7f7fe640 1 -- 172.21.15.5:0/743697575 learned_addr learned my addr 172.21.15.5:0/743697575 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:31.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.828+0000 7f3d7effd640 1 --2- 172.21.15.5:0/743697575 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d6c004050 0x7f3d800b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:31.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.828+0000 7f3d77fff640 1 --2- 172.21.15.5:0/743697575 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d80145e10 0x7f3d80146c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:31.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.828+0000 7f3d7f7fe640 1 -- 172.21.15.5:0/743697575 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d6c004050 msgr2=0x7f3d800b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:31.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.828+0000 7f3d7f7fe640 1 --2- 172.21.15.5:0/743697575 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d6c004050 0x7f3d800b03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:31.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.828+0000 7f3d7f7fe640 1 -- 172.21.15.5:0/743697575 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d80145e10 msgr2=0x7f3d80146c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:31.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.828+0000 7f3d7f7fe640 1 --2- 172.21.15.5:0/743697575 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d80145e10 0x7f3d80146c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:31.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.828+0000 7f3d7f7fe640 1 -- 172.21.15.5:0/743697575 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d8013fb50 con 0x7f3d80150f80 2024-03-18T15:34:31.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.828+0000 7f3d7f7fe640 1 --2- 172.21.15.5:0/743697575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d80150f80 0x7f3d800ad8e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3d8014bac0 tx=0x7f3d7800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:31.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.828+0000 7f3d777fe640 1 -- 172.21.15.5:0/743697575 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7800bb80 con 0x7f3d80150f80 2024-03-18T15:34:31.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.828+0000 7f3d777fe640 1 -- 172.21.15.5:0/743697575 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d7800bd20 con 0x7f3d80150f80 2024-03-18T15:34:31.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.828+0000 7f3d857f6640 1 -- 172.21.15.5:0/743697575 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d80151430 con 0x7f3d80150f80 2024-03-18T15:34:31.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.828+0000 7f3d777fe640 1 -- 172.21.15.5:0/743697575 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d78007330 con 0x7f3d80150f80 2024-03-18T15:34:31.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.828+0000 7f3d857f6640 1 -- 172.21.15.5:0/743697575 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d801464f0 con 0x7f3d80150f80 2024-03-18T15:34:31.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.828+0000 7f3d857f6640 1 -- 172.21.15.5:0/743697575 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d8004f550 con 0x7f3d80150f80 2024-03-18T15:34:31.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.828+0000 7f3d777fe640 1 -- 172.21.15.5:0/743697575 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f3d7801b060 con 0x7f3d80150f80 2024-03-18T15:34:31.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.828+0000 7f3d777fe640 1 --2- 172.21.15.5:0/743697575 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3d60067660 0x7f3d60069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:31.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.828+0000 7f3d7effd640 1 --2- 172.21.15.5:0/743697575 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3d60067660 0x7f3d60069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:31.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.828+0000 7f3d777fe640 1 -- 172.21.15.5:0/743697575 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7f3d780858d0 con 0x7f3d80150f80 2024-03-18T15:34:31.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.832+0000 7f3d7effd640 1 --2- 172.21.15.5:0/743697575 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3d60067660 0x7f3d60069b20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3d70005510 tx=0x7f3d70009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:31.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:31.832+0000 7f3d777fe640 1 -- 172.21.15.5:0/743697575 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f3d80151430 con 0x7f3d80150f80 2024-03-18T15:34:32.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.068+0000 7f3d857f6640 1 -- 172.21.15.5:0/743697575 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3d8014db30 con 0x7f3d80150f80 2024-03-18T15:34:32.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.072+0000 7f3d777fe640 1 -- 172.21.15.5:0/743697575 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) ==== 74+0+16513 (secure 0 0 0) 0x7f3d780565e0 con 0x7f3d80150f80 2024-03-18T15:34:32.076 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:34:32.076 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":30,"fsid":"ef54fe03-5aa7-4eb2-9b1e-66c9cc345b94","created":"2024-03-18T15:33:18.086614+0000","modified":"2024-03-18T15:34:23.736672+0000","last_up_change":"2024-03-18T15:33:29.229911+0000","last_in_change":"2024-03-18T15:33:20.323840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:28.963186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-18T15:34:20.652674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-18T15:34:21.110594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df1c754b-af59-4d16-8919-fbb6386b3c48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6804","nonce":976264754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6808","nonce":976264754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6815","nonce":976264754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6811","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6813","nonce":976264754}]},"public_addr":"172.21.15.5:6804/976264754","cluster_addr":"172.21.15.5:6808/976264754","heartbeat_back_addr":"172.21.15.5:6815/976264754","heartbeat_front_addr":"172.21.15.5:6813/976264754","state":["exists","up"]},{"osd":1,"uuid":"b9a08b5e-8af4-4144-bdba-d69ed1ac627a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6801","nonce":3901761126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6803","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6805","nonce":3901761126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6812","nonce":3901761126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6807","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6809","nonce":3901761126}]},"public_addr":"172.21.15.5:6801/3901761126","cluster_addr":"172.21.15.5:6805/3901761126","heartbeat_back_addr":"172.21.15.5:6812/3901761126","heartbeat_front_addr":"172.21.15.5:6809/3901761126","state":["exists","up"]},{"osd":2,"uuid":"c3e8bb76-e382-439d-b380-62ef92f1255a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6817","nonce":502968448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6819","nonce":502968448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6823","nonce":502968448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6821","nonce":502968448}]},"public_addr":"172.21.15.5:6817/502968448","cluster_addr":"172.21.15.5:6819/502968448","heartbeat_back_addr":"172.21.15.5:6823/502968448","heartbeat_front_addr":"172.21.15.5:6821/502968448","state":["exists","up"]},{"osd":3,"uuid":"e8c94625-f2cd-4eba-b53d-2e7356e1c02a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6825","nonce":2425932735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6827","nonce":2425932735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6831","nonce":2425932735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6829","nonce":2425932735}]},"public_addr":"172.21.15.5:6825/2425932735","cluster_addr":"172.21.15.5:6827/2425932735","heartbeat_back_addr":"172.21.15.5:6831/2425932735","heartbeat_front_addr":"172.21.15.5:6829/2425932735","state":["exists","up"]},{"osd":4,"uuid":"0099dd52-6265-4d32-82e4-3ef16e73add0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6817","nonce":976514285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6819","nonce":976514285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6823","nonce":976514285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6821","nonce":976514285}]},"public_addr":"172.21.15.80:6817/976514285","cluster_addr":"172.21.15.80:6819/976514285","heartbeat_back_addr":"172.21.15.80:6823/976514285","heartbeat_front_addr":"172.21.15.80:6821/976514285","state":["exists","up"]},{"osd":5,"uuid":"ed10c84f-8c39-41cc-9b64-7c482a94a48e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6825","nonce":627153447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6827","nonce":627153447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6831","nonce":627153447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6829","nonce":627153447}]},"public_addr":"172.21.15.80:6825/627153447","cluster_addr":"172.21.15.80:6827/627153447","heartbeat_back_addr":"172.21.15.80:6831/627153447","heartbeat_front_addr":"172.21.15.80:6829/627153447","state":["exists","up"]},{"osd":6,"uuid":"c7a3538a-7048-4fba-8781-b440f03c25db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6803","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6805","nonce":488342527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6807","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6809","nonce":488342527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6815","nonce":488342527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6811","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6813","nonce":488342527}]},"public_addr":"172.21.15.80:6805/488342527","cluster_addr":"172.21.15.80:6809/488342527","heartbeat_back_addr":"172.21.15.80:6815/488342527","heartbeat_front_addr":"172.21.15.80:6813/488342527","state":["exists","up"]},{"osd":7,"uuid":"f9418104-3a97-4110-bc3a-1f8151d0f757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6801","nonce":982831666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6804","nonce":982831666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6812","nonce":982831666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6808","nonce":982831666}]},"public_addr":"172.21.15.80:6801/982831666","cluster_addr":"172.21.15.80:6804/982831666","heartbeat_back_addr":"172.21.15.80:6812/982831666","heartbeat_front_addr":"172.21.15.80:6808/982831666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.970828+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.858116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.987556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.864603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.756293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.591502+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:34:32.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.076+0000 7f3d857f6640 1 -- 172.21.15.5:0/743697575 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3d60067660 msgr2=0x7f3d60069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:32.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.076+0000 7f3d857f6640 1 --2- 172.21.15.5:0/743697575 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3d60067660 0x7f3d60069b20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3d70005510 tx=0x7f3d70009040 comp rx=0 tx=0).stop 2024-03-18T15:34:32.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.076+0000 7f3d857f6640 1 -- 172.21.15.5:0/743697575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d80150f80 msgr2=0x7f3d800ad8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:32.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.076+0000 7f3d857f6640 1 --2- 172.21.15.5:0/743697575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d80150f80 0x7f3d800ad8e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3d8014bac0 tx=0x7f3d7800b4a0 comp rx=0 tx=0).stop 2024-03-18T15:34:32.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.076+0000 7f3d857f6640 1 -- 172.21.15.5:0/743697575 shutdown_connections 2024-03-18T15:34:32.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.076+0000 7f3d857f6640 1 --2- 172.21.15.5:0/743697575 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3d60067660 0x7f3d60069b20 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:32.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.076+0000 7f3d857f6640 1 --2- 172.21.15.5:0/743697575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d80150f80 0x7f3d800ad8e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:32.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.076+0000 7f3d857f6640 1 --2- 172.21.15.5:0/743697575 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d80145e10 0x7f3d80146c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:32.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.076+0000 7f3d857f6640 1 --2- 172.21.15.5:0/743697575 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3d6c004050 0x7f3d800b03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:32.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.076+0000 7f3d857f6640 1 -- 172.21.15.5:0/743697575 >> 172.21.15.5:0/743697575 conn(0x7f3d800b6680 msgr2=0x7f3d80141560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:32.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.076+0000 7f3d857f6640 1 -- 172.21.15.5:0/743697575 shutdown_connections 2024-03-18T15:34:32.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.076+0000 7f3d857f6640 1 -- 172.21.15.5:0/743697575 wait complete. 2024-03-18T15:34:32.095 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T15:33:28.963186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-18T15:34:20.652674+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '29', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-03-18T15:34:21.110594+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '30', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T15:34:32.095 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T15:34:32.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.224+0000 7fd00fb92640 1 Processor -- start 2024-03-18T15:34:32.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.224+0000 7fd00fb92640 1 -- start start 2024-03-18T15:34:32.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.224+0000 7fd00fb92640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd008144aa0 0x7fd0080b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:32.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.224+0000 7fd00fb92640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd008141420 0x7fd008141890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:32.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.224+0000 7fd00fb92640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd008141dd0 con 0x7fd0080744b0 2024-03-18T15:34:32.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.224+0000 7fd00fb92640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fd00806ff20 con 0x7fd008141420 2024-03-18T15:34:32.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.224+0000 7fd00fb92640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fd008070170 con 0x7fd008144aa0 2024-03-18T15:34:32.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.224+0000 7fd00d92e640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd0080744b0 0x7fd008144390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44276/0 (socket says 172.21.15.5:44276) 2024-03-18T15:34:32.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.224+0000 7fd00d92e640 1 -- 172.21.15.5:0/2427244715 learned_addr learned my addr 172.21.15.5:0/2427244715 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:32.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.224+0000 7fd00e12f640 1 --2- 172.21.15.5:0/2427244715 >> v2:172.21.15.5:3300/0 conn(0x7fd008141420 0x7fd008141890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:32.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.224+0000 7fd00d12d640 1 --2- 172.21.15.5:0/2427244715 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd008144aa0 0x7fd0080b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:32.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.224+0000 7fd00e12f640 1 -- 172.21.15.5:0/2427244715 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd008144aa0 msgr2=0x7fd0080b53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:32.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.224+0000 7fd00e12f640 1 --2- 172.21.15.5:0/2427244715 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd008144aa0 0x7fd0080b53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:32.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.224+0000 7fd00e12f640 1 -- 172.21.15.5:0/2427244715 >> v1:172.21.15.5:6789/0 conn(0x7fd0080744b0 legacy=0x7fd008144390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:32.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.224+0000 7fd00e12f640 1 -- 172.21.15.5:0/2427244715 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd008074890 con 0x7fd008141420 2024-03-18T15:34:32.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.224+0000 7fd00c92c640 1 -- 172.21.15.5:0/2427244715 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3157796415 0 0) 0x7fd008141dd0 con 0x7fd0080744b0 2024-03-18T15:34:32.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.224+0000 7fd00d12d640 1 --2- 172.21.15.5:0/2427244715 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd008144aa0 0x7fd0080b53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:32.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.228+0000 7fd00e12f640 1 --2- 172.21.15.5:0/2427244715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd008141420 0x7fd008141890 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd0040097c0 tx=0x7fd00401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1627ff5d5f3d01e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:32.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.228+0000 7fd00c92c640 1 -- 172.21.15.5:0/2427244715 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0040b1070 con 0x7fd008141420 2024-03-18T15:34:32.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.228+0000 7fd00c92c640 1 -- 172.21.15.5:0/2427244715 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0040a8050 con 0x7fd008141420 2024-03-18T15:34:32.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.228+0000 7fd00c92c640 1 -- 172.21.15.5:0/2427244715 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0040ac860 con 0x7fd008141420 2024-03-18T15:34:32.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.228+0000 7fd00fb92640 1 -- 172.21.15.5:0/2427244715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd008141420 msgr2=0x7fd008141890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:32.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.228+0000 7fd00fb92640 1 --2- 172.21.15.5:0/2427244715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd008141420 0x7fd008141890 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd0040097c0 tx=0x7fd00401c3e0 comp rx=0 tx=0).stop 2024-03-18T15:34:32.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.228+0000 7fd00fb92640 1 -- 172.21.15.5:0/2427244715 shutdown_connections 2024-03-18T15:34:32.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.228+0000 7fd00fb92640 1 --2- 172.21.15.5:0/2427244715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd008141420 0x7fd008141890 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:32.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.228+0000 7fd00fb92640 1 --2- 172.21.15.5:0/2427244715 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd008144aa0 0x7fd0080b53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:32.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.228+0000 7fd00fb92640 1 -- 172.21.15.5:0/2427244715 >> 172.21.15.5:0/2427244715 conn(0x7fd0080b9190 msgr2=0x7fd0080b95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:32.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.228+0000 7fd00fb92640 1 -- 172.21.15.5:0/2427244715 shutdown_connections 2024-03-18T15:34:32.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.228+0000 7fd00fb92640 1 -- 172.21.15.5:0/2427244715 wait complete. 2024-03-18T15:34:32.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.228+0000 7fd00fb92640 1 Processor -- start 2024-03-18T15:34:32.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.228+0000 7fd00fb92640 1 -- start start 2024-03-18T15:34:32.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.228+0000 7fd00fb92640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd0080744b0 0x7fd00813fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:32.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.228+0000 7fd00fb92640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd008141420 0x7fd0081e0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:32.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.228+0000 7fd00fb92640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd008144aa0 0x7fd0080a9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:32.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.232+0000 7fd00fb92640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd008151270 con 0x7fd008141420 2024-03-18T15:34:32.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.232+0000 7fd00fb92640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd0080b4c20 con 0x7fd008144aa0 2024-03-18T15:34:32.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.232+0000 7fd00fb92640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fd0081513f0 con 0x7fd0080744b0 2024-03-18T15:34:32.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.232+0000 7fd00d12d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd008141420 0x7fd0081e0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:32.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.232+0000 7fd00d12d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd008141420 0x7fd0081e0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56404/0 (socket says 172.21.15.5:56404) 2024-03-18T15:34:32.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.232+0000 7fd00d12d640 1 -- 172.21.15.5:0/2108239517 learned_addr learned my addr 172.21.15.5:0/2108239517 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:32.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.232+0000 7fd00d92e640 1 --2- 172.21.15.5:0/2108239517 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd0080744b0 0x7fd00813fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:32.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.232+0000 7fd00d12d640 1 -- 172.21.15.5:0/2108239517 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd0080744b0 msgr2=0x7fd00813fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:32.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.232+0000 7fd00e12f640 1 --2- 172.21.15.5:0/2108239517 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd008144aa0 0x7fd0080a9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:32.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.232+0000 7fd00d12d640 1 --2- 172.21.15.5:0/2108239517 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd0080744b0 0x7fd00813fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:32.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.232+0000 7fd00d12d640 1 -- 172.21.15.5:0/2108239517 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd008144aa0 msgr2=0x7fd0080a9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:32.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.232+0000 7fd00d12d640 1 --2- 172.21.15.5:0/2108239517 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd008144aa0 0x7fd0080a9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:32.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.232+0000 7fd00d12d640 1 -- 172.21.15.5:0/2108239517 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd00814e720 con 0x7fd008141420 2024-03-18T15:34:32.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.232+0000 7fd00d12d640 1 --2- 172.21.15.5:0/2108239517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd008141420 0x7fd0081e0960 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fcff800c7c0 tx=0x7fcff800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:32.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.232+0000 7fcffe7fc640 1 -- 172.21.15.5:0/2108239517 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff8007a30 con 0x7fd008141420 2024-03-18T15:34:32.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.232+0000 7fcffe7fc640 1 -- 172.21.15.5:0/2108239517 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcff8007bd0 con 0x7fd008141420 2024-03-18T15:34:32.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.232+0000 7fd00fb92640 1 -- 172.21.15.5:0/2108239517 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd008073480 con 0x7fd008141420 2024-03-18T15:34:32.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.232+0000 7fcffe7fc640 1 -- 172.21.15.5:0/2108239517 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcff80113f0 con 0x7fd008141420 2024-03-18T15:34:32.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.232+0000 7fd00fb92640 1 -- 172.21.15.5:0/2108239517 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0081e0ea0 con 0x7fd008141420 2024-03-18T15:34:32.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.232+0000 7fd00fb92640 1 -- 172.21.15.5:0/2108239517 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd00804f550 con 0x7fd008141420 2024-03-18T15:34:32.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.232+0000 7fcffe7fc640 1 -- 172.21.15.5:0/2108239517 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fcff8011590 con 0x7fd008141420 2024-03-18T15:34:32.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.232+0000 7fcffe7fc640 1 --2- 172.21.15.5:0/2108239517 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fcfec067660 0x7fcfec069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:32.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.232+0000 7fcffe7fc640 1 -- 172.21.15.5:0/2108239517 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7fcff8085d90 con 0x7fd008141420 2024-03-18T15:34:32.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.232+0000 7fd00d92e640 1 --2- 172.21.15.5:0/2108239517 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fcfec067660 0x7fcfec069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:32.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.232+0000 7fd00d92e640 1 --2- 172.21.15.5:0/2108239517 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fcfec067660 0x7fcfec069b20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd0000012f0 tx=0x7fd000009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:32.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.236+0000 7fcffe7fc640 1 -- 172.21.15.5:0/2108239517 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fcff8015020 con 0x7fd008141420 2024-03-18T15:34:32.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.396+0000 7fd00fb92640 1 -- 172.21.15.5:0/2108239517 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd00813c9c0 con 0x7fd008141420 2024-03-18T15:34:32.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.396+0000 7fcffe7fc640 1 -- 172.21.15.5:0/2108239517 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) ==== 93+0+10 (secure 0 0 0) 0x7fcff8056aa0 con 0x7fd008141420 2024-03-18T15:34:32.399 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-03-18T15:34:32.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.396+0000 7fd00fb92640 1 -- 172.21.15.5:0/2108239517 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fcfec067660 msgr2=0x7fcfec069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:32.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.396+0000 7fd00fb92640 1 --2- 172.21.15.5:0/2108239517 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fcfec067660 0x7fcfec069b20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd0000012f0 tx=0x7fd000009040 comp rx=0 tx=0).stop 2024-03-18T15:34:32.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.396+0000 7fd00fb92640 1 -- 172.21.15.5:0/2108239517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd008141420 msgr2=0x7fd0081e0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:32.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.396+0000 7fd00fb92640 1 --2- 172.21.15.5:0/2108239517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd008141420 0x7fd0081e0960 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fcff800c7c0 tx=0x7fcff800cc80 comp rx=0 tx=0).stop 2024-03-18T15:34:32.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.400+0000 7fd00fb92640 1 -- 172.21.15.5:0/2108239517 shutdown_connections 2024-03-18T15:34:32.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.400+0000 7fd00fb92640 1 --2- 172.21.15.5:0/2108239517 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fcfec067660 0x7fcfec069b20 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:32.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.400+0000 7fd00fb92640 1 --2- 172.21.15.5:0/2108239517 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd008144aa0 0x7fd0080a9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:32.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.400+0000 7fd00fb92640 1 --2- 172.21.15.5:0/2108239517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd008141420 0x7fd0081e0960 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:32.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.400+0000 7fd00fb92640 1 --2- 172.21.15.5:0/2108239517 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd0080744b0 0x7fd00813fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:32.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.400+0000 7fd00fb92640 1 -- 172.21.15.5:0/2108239517 >> 172.21.15.5:0/2108239517 conn(0x7fd0080b9190 msgr2=0x7fd00813cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:32.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.400+0000 7fd00fb92640 1 -- 172.21.15.5:0/2108239517 shutdown_connections 2024-03-18T15:34:32.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.400+0000 7fd00fb92640 1 -- 172.21.15.5:0/2108239517 wait complete. 2024-03-18T15:34:32.417 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-18T15:34:32.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.532+0000 7ff27c346640 1 Processor -- start 2024-03-18T15:34:32.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.532+0000 7ff27c346640 1 -- start start 2024-03-18T15:34:32.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.532+0000 7ff27c346640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff27406e390 0x7ff274070360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:32.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.532+0000 7ff27c346640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff274148130 0x7ff2741416c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:32.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.532+0000 7ff27c346640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff27406e770 con 0x7ff274141260 2024-03-18T15:34:32.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.532+0000 7ff27c346640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7ff274072760 con 0x7ff274148130 2024-03-18T15:34:32.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.532+0000 7ff27c346640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7ff274072a40 con 0x7ff27406e390 2024-03-18T15:34:32.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.532+0000 7ff2798e1640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7ff274141260 0x7ff274147a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44282/0 (socket says 172.21.15.5:44282) 2024-03-18T15:34:32.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.532+0000 7ff2798e1640 1 -- 172.21.15.5:0/363498546 learned_addr learned my addr 172.21.15.5:0/363498546 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:32.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.532+0000 7ff27a0e2640 1 --2- 172.21.15.5:0/363498546 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff27406e390 0x7ff274070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:32.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.532+0000 7ff27a8e3640 1 --2- 172.21.15.5:0/363498546 >> v2:172.21.15.80:3300/0 conn(0x7ff274148130 0x7ff2741416c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:32.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.532+0000 7ff2790e0640 1 -- 172.21.15.5:0/363498546 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1215391369 0 0) 0x7ff27406e770 con 0x7ff274141260 2024-03-18T15:34:32.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.532+0000 7ff2790e0640 1 -- 172.21.15.5:0/363498546 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff260003530 con 0x7ff274141260 2024-03-18T15:34:32.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.532+0000 7ff2790e0640 1 -- 172.21.15.5:0/363498546 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 14977121 0 0) 0x7ff260003530 con 0x7ff274141260 2024-03-18T15:34:32.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.532+0000 7ff2790e0640 1 -- 172.21.15.5:0/363498546 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff27406e390 msgr2=0x7ff274070360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:32.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.532+0000 7ff2790e0640 1 --2- 172.21.15.5:0/363498546 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff27406e390 0x7ff274070360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:32.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.532+0000 7ff2790e0640 1 -- 172.21.15.5:0/363498546 >> v2:172.21.15.80:3300/0 conn(0x7ff274148130 msgr2=0x7ff2741416c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:32.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.532+0000 7ff2790e0640 1 --2- 172.21.15.5:0/363498546 >> v2:172.21.15.80:3300/0 conn(0x7ff274148130 0x7ff2741416c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:32.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.532+0000 7ff2790e0640 1 -- 172.21.15.5:0/363498546 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2740713e0 con 0x7ff274141260 2024-03-18T15:34:32.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.532+0000 7ff2790e0640 1 -- 172.21.15.5:0/363498546 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3837549003 0 0) 0x7ff264001680 con 0x7ff274141260 2024-03-18T15:34:32.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.532+0000 7ff2790e0640 1 -- 172.21.15.5:0/363498546 >> v1:172.21.15.5:6789/0 conn(0x7ff274141260 legacy=0x7ff274147a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:32.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.532+0000 7ff2790e0640 1 --2- 172.21.15.5:0/363498546 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff260003aa0 0x7ff260003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:32.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.532+0000 7ff2790e0640 1 --2- 172.21.15.5:0/363498546 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff27406e390 0x7ff260004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:32.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.532+0000 7ff2790e0640 1 --2- 172.21.15.5:0/363498546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff260004d50 0x7ff2600051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:32.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.532+0000 7ff2790e0640 1 -- 172.21.15.5:0/363498546 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff274072760 con 0x7ff260004d50 2024-03-18T15:34:32.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.532+0000 7ff2790e0640 1 -- 172.21.15.5:0/363498546 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff274072a40 con 0x7ff260003aa0 2024-03-18T15:34:32.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.532+0000 7ff2790e0640 1 -- 172.21.15.5:0/363498546 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7ff26000baa0 con 0x7ff27406e390 2024-03-18T15:34:32.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.532+0000 7ff27a8e3640 1 --2- 172.21.15.5:0/363498546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff260004d50 0x7ff2600051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:32.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.532+0000 7ff27a0e2640 1 --2- 172.21.15.5:0/363498546 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff260003aa0 0x7ff260003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:32.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.532+0000 7ff2798e1640 1 --2- 172.21.15.5:0/363498546 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff27406e390 0x7ff260004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:32.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.532+0000 7ff27a8e3640 1 -- 172.21.15.5:0/363498546 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff27406e390 msgr2=0x7ff260004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:32.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.532+0000 7ff27a8e3640 1 --2- 172.21.15.5:0/363498546 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff27406e390 0x7ff260004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:32.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.532+0000 7ff27a8e3640 1 -- 172.21.15.5:0/363498546 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff260003aa0 msgr2=0x7ff260003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:32.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.532+0000 7ff27a8e3640 1 --2- 172.21.15.5:0/363498546 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff260003aa0 0x7ff260003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:32.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.532+0000 7ff27a8e3640 1 -- 172.21.15.5:0/363498546 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff260003530 con 0x7ff260004d50 2024-03-18T15:34:32.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.536+0000 7ff27a8e3640 1 --2- 172.21.15.5:0/363498546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff260004d50 0x7ff2600051d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff26c009fc0 tx=0x7ff26c01ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a67494adc5ca7a95 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:32.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.536+0000 7ff2790e0640 1 -- 172.21.15.5:0/363498546 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26c0b2070 con 0x7ff260004d50 2024-03-18T15:34:32.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.536+0000 7ff2790e0640 1 -- 172.21.15.5:0/363498546 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff26c0a5bc0 con 0x7ff260004d50 2024-03-18T15:34:32.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.536+0000 7ff2790e0640 1 -- 172.21.15.5:0/363498546 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26c009c10 con 0x7ff260004d50 2024-03-18T15:34:32.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.536+0000 7ff27c346640 1 -- 172.21.15.5:0/363498546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff260004d50 msgr2=0x7ff2600051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:32.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.536+0000 7ff27c346640 1 --2- 172.21.15.5:0/363498546 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff260004d50 0x7ff2600051d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff26c009fc0 tx=0x7ff26c01ccc0 comp rx=0 tx=0).stop 2024-03-18T15:34:32.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.536+0000 7ff27a8e3640 1 -- 172.21.15.5:0/363498546 reap_dead start 2024-03-18T15:34:32.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.536+0000 7ff27c346640 1 -- 172.21.15.5:0/363498546 shutdown_connections 2024-03-18T15:34:32.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.536+0000 7ff27c346640 1 -- 172.21.15.5:0/363498546 >> 172.21.15.5:0/363498546 conn(0x7ff2740b6680 msgr2=0x7ff2740b6a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:32.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.536+0000 7ff27c346640 1 -- 172.21.15.5:0/363498546 shutdown_connections 2024-03-18T15:34:32.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.536+0000 7ff27c346640 1 -- 172.21.15.5:0/363498546 wait complete. 2024-03-18T15:34:32.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.536+0000 7ff27c346640 1 Processor -- start 2024-03-18T15:34:32.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.536+0000 7ff27c346640 1 -- start start 2024-03-18T15:34:32.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.536+0000 7ff27c346640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff260004d50 0x7ff2741d3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:32.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.536+0000 7ff27c346640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff274141d30 0x7ff2741444f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:32.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.536+0000 7ff27c346640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff274144a30 0x7ff27413cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:32.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.536+0000 7ff27c346640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff274070360 con 0x7ff274141d30 2024-03-18T15:34:32.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.536+0000 7ff27c346640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff2740719a0 con 0x7ff260004d50 2024-03-18T15:34:32.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.536+0000 7ff27c346640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7ff2740704e0 con 0x7ff274144a30 2024-03-18T15:34:32.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.536+0000 7ff2798e1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff274141d30 0x7ff2741444f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:32.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.536+0000 7ff2798e1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff274141d30 0x7ff2741444f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56428/0 (socket says 172.21.15.5:56428) 2024-03-18T15:34:32.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.536+0000 7ff2798e1640 1 -- 172.21.15.5:0/407441003 learned_addr learned my addr 172.21.15.5:0/407441003 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:32.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.536+0000 7ff27a0e2640 1 --2- 172.21.15.5:0/407441003 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff260004d50 0x7ff2741d3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:32.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.536+0000 7ff27a8e3640 1 --2- 172.21.15.5:0/407441003 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff274144a30 0x7ff27413cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:32.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.540+0000 7ff2798e1640 1 -- 172.21.15.5:0/407441003 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff274144a30 msgr2=0x7ff27413cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:32.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.540+0000 7ff2798e1640 1 --2- 172.21.15.5:0/407441003 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff274144a30 0x7ff27413cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:32.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.540+0000 7ff2798e1640 1 -- 172.21.15.5:0/407441003 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff260004d50 msgr2=0x7ff2741d3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:32.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.540+0000 7ff2798e1640 1 --2- 172.21.15.5:0/407441003 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff260004d50 0x7ff2741d3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:32.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.540+0000 7ff2798e1640 1 -- 172.21.15.5:0/407441003 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff274144e70 con 0x7ff274141d30 2024-03-18T15:34:32.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.540+0000 7ff2798e1640 1 --2- 172.21.15.5:0/407441003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff274141d30 0x7ff2741444f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff2640117f0 tx=0x7ff264011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:32.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.540+0000 7ff26affd640 1 -- 172.21.15.5:0/407441003 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff264010040 con 0x7ff274141d30 2024-03-18T15:34:32.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.540+0000 7ff26affd640 1 -- 172.21.15.5:0/407441003 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff26401a070 con 0x7ff274141d30 2024-03-18T15:34:32.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.540+0000 7ff27c346640 1 -- 172.21.15.5:0/407441003 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff274143bd0 con 0x7ff274141d30 2024-03-18T15:34:32.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.540+0000 7ff26affd640 1 -- 172.21.15.5:0/407441003 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff26400fd80 con 0x7ff274141d30 2024-03-18T15:34:32.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.540+0000 7ff27c346640 1 -- 172.21.15.5:0/407441003 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2741428d0 con 0x7ff274141d30 2024-03-18T15:34:32.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.540+0000 7ff27c346640 1 -- 172.21.15.5:0/407441003 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2741423e0 con 0x7ff274141d30 2024-03-18T15:34:32.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.540+0000 7ff26affd640 1 -- 172.21.15.5:0/407441003 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7ff26401f460 con 0x7ff274141d30 2024-03-18T15:34:32.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.540+0000 7ff26affd640 1 --2- 172.21.15.5:0/407441003 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7ff258067660 0x7ff258069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:32.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.540+0000 7ff26affd640 1 -- 172.21.15.5:0/407441003 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7ff264089390 con 0x7ff274141d30 2024-03-18T15:34:32.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.540+0000 7ff27a0e2640 1 --2- 172.21.15.5:0/407441003 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7ff258067660 0x7ff258069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:32.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.540+0000 7ff27a0e2640 1 --2- 172.21.15.5:0/407441003 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7ff258067660 0x7ff258069b20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff270009be0 tx=0x7ff270009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:32.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.552+0000 7ff26affd640 1 -- 172.21.15.5:0/407441003 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7ff26405a080 con 0x7ff274141d30 2024-03-18T15:34:32.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.708+0000 7ff27c346640 1 -- 172.21.15.5:0/407441003 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff27404f550 con 0x7ff274141d30 2024-03-18T15:34:32.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.708+0000 7ff26affd640 1 -- 172.21.15.5:0/407441003 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) ==== 104+0+11 (secure 0 0 0) 0x7ff264059980 con 0x7ff274141d30 2024-03-18T15:34:32.712 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-03-18T15:34:32.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.712+0000 7ff27c346640 1 -- 172.21.15.5:0/407441003 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7ff258067660 msgr2=0x7ff258069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:32.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.712+0000 7ff27c346640 1 --2- 172.21.15.5:0/407441003 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7ff258067660 0x7ff258069b20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff270009be0 tx=0x7ff270009190 comp rx=0 tx=0).stop 2024-03-18T15:34:32.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.712+0000 7ff27c346640 1 -- 172.21.15.5:0/407441003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff274141d30 msgr2=0x7ff2741444f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:32.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.712+0000 7ff27c346640 1 --2- 172.21.15.5:0/407441003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff274141d30 0x7ff2741444f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff2640117f0 tx=0x7ff264011bf0 comp rx=0 tx=0).stop 2024-03-18T15:34:32.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.712+0000 7ff27c346640 1 -- 172.21.15.5:0/407441003 shutdown_connections 2024-03-18T15:34:32.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.712+0000 7ff27c346640 1 --2- 172.21.15.5:0/407441003 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7ff258067660 0x7ff258069b20 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:32.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.712+0000 7ff27c346640 1 --2- 172.21.15.5:0/407441003 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7ff274144a30 0x7ff27413cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:32.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.712+0000 7ff27c346640 1 --2- 172.21.15.5:0/407441003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff274141d30 0x7ff2741444f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:32.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.712+0000 7ff27c346640 1 --2- 172.21.15.5:0/407441003 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff260004d50 0x7ff2741d3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:32.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.712+0000 7ff27c346640 1 -- 172.21.15.5:0/407441003 >> 172.21.15.5:0/407441003 conn(0x7ff2740b6680 msgr2=0x7ff2740b8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:32.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.712+0000 7ff27c346640 1 -- 172.21.15.5:0/407441003 shutdown_connections 2024-03-18T15:34:32.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.712+0000 7ff27c346640 1 -- 172.21.15.5:0/407441003 wait complete. 2024-03-18T15:34:32.735 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-18T15:34:32.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.856+0000 7f6a4deda640 1 Processor -- start 2024-03-18T15:34:32.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.856+0000 7f6a4deda640 1 -- start start 2024-03-18T15:34:32.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.856+0000 7f6a4deda640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f6a480b51b0 0x7f6a480b6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:32.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.856+0000 7f6a4deda640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6a480b4430 0x7f6a481472a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:32.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.856+0000 7f6a4deda640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a480b5590 con 0x7f6a481477e0 2024-03-18T15:34:32.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.856+0000 7f6a4deda640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f6a48147c90 con 0x7f6a480b51b0 2024-03-18T15:34:32.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.856+0000 7f6a4deda640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f6a480b97e0 con 0x7f6a480b4430 2024-03-18T15:34:32.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.856+0000 7f6a47fff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f6a481477e0 0x7f6a48140ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44284/0 (socket says 172.21.15.5:44284) 2024-03-18T15:34:32.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.856+0000 7f6a47fff640 1 -- 172.21.15.5:0/2534838397 learned_addr learned my addr 172.21.15.5:0/2534838397 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:32.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.856+0000 7f6a477fe640 1 --2- 172.21.15.5:0/2534838397 >> v2:172.21.15.80:3300/0 conn(0x7f6a480b51b0 0x7f6a480b6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:32.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.856+0000 7f6a46ffd640 1 --2- 172.21.15.5:0/2534838397 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6a480b4430 0x7f6a481472a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:32.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.856+0000 7f6a467fc640 1 -- 172.21.15.5:0/2534838397 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 657618176 0 0) 0x7f6a480b5590 con 0x7f6a481477e0 2024-03-18T15:34:32.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.856+0000 7f6a467fc640 1 -- 172.21.15.5:0/2534838397 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a30003530 con 0x7f6a481477e0 2024-03-18T15:34:32.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.856+0000 7f6a467fc640 1 -- 172.21.15.5:0/2534838397 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3983555601 0 0) 0x7f6a30003530 con 0x7f6a481477e0 2024-03-18T15:34:32.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.856+0000 7f6a467fc640 1 -- 172.21.15.5:0/2534838397 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6a480b4430 msgr2=0x7f6a481472a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:32.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.856+0000 7f6a467fc640 1 --2- 172.21.15.5:0/2534838397 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6a480b4430 0x7f6a481472a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:32.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.856+0000 7f6a467fc640 1 -- 172.21.15.5:0/2534838397 >> v2:172.21.15.80:3300/0 conn(0x7f6a480b51b0 msgr2=0x7f6a480b6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:32.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.856+0000 7f6a467fc640 1 --2- 172.21.15.5:0/2534838397 >> v2:172.21.15.80:3300/0 conn(0x7f6a480b51b0 0x7f6a480b6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:32.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.856+0000 7f6a467fc640 1 -- 172.21.15.5:0/2534838397 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a480b9ac0 con 0x7f6a481477e0 2024-03-18T15:34:32.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.856+0000 7f6a467fc640 1 -- 172.21.15.5:0/2534838397 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3837549003 0 0) 0x7f6a40001a00 con 0x7f6a481477e0 2024-03-18T15:34:32.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.856+0000 7f6a467fc640 1 -- 172.21.15.5:0/2534838397 >> v1:172.21.15.5:6789/0 conn(0x7f6a481477e0 legacy=0x7f6a48140ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:32.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.856+0000 7f6a467fc640 1 --2- 172.21.15.5:0/2534838397 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6a480b4430 0x7f6a30003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:32.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.856+0000 7f6a467fc640 1 --2- 172.21.15.5:0/2534838397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a30004020 0x7f6a30004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:32.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.856+0000 7f6a467fc640 1 --2- 172.21.15.5:0/2534838397 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6a30004d50 0x7f6a300051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:32.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.856+0000 7f6a467fc640 1 -- 172.21.15.5:0/2534838397 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6a48147c90 con 0x7f6a30004020 2024-03-18T15:34:32.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.856+0000 7f6a467fc640 1 -- 172.21.15.5:0/2534838397 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f6a480b97e0 con 0x7f6a30004d50 2024-03-18T15:34:32.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.856+0000 7f6a467fc640 1 -- 172.21.15.5:0/2534838397 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f6a3000ba50 con 0x7f6a480b4430 2024-03-18T15:34:32.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.856+0000 7f6a46ffd640 1 --2- 172.21.15.5:0/2534838397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a30004020 0x7f6a30004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:32.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.856+0000 7f6a477fe640 1 --2- 172.21.15.5:0/2534838397 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6a480b4430 0x7f6a30003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:32.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.856+0000 7f6a47fff640 1 --2- 172.21.15.5:0/2534838397 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6a30004d50 0x7f6a300051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:32.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.856+0000 7f6a46ffd640 1 -- 172.21.15.5:0/2534838397 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6a480b4430 msgr2=0x7f6a30003aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:32.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.856+0000 7f6a46ffd640 1 --2- 172.21.15.5:0/2534838397 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6a480b4430 0x7f6a30003aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:32.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.856+0000 7f6a46ffd640 1 -- 172.21.15.5:0/2534838397 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6a30004d50 msgr2=0x7f6a300051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:32.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.856+0000 7f6a46ffd640 1 --2- 172.21.15.5:0/2534838397 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6a30004d50 0x7f6a300051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:32.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.856+0000 7f6a46ffd640 1 -- 172.21.15.5:0/2534838397 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a30003530 con 0x7f6a30004020 2024-03-18T15:34:32.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.856+0000 7f6a47fff640 1 --2- 172.21.15.5:0/2534838397 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6a30004d50 0x7f6a300051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:32.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.860+0000 7f6a46ffd640 1 --2- 172.21.15.5:0/2534838397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a30004020 0x7f6a30004810 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c009ac0 tx=0x7f6a3c01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba23c65b9cdb36b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:32.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.860+0000 7f6a467fc640 1 -- 172.21.15.5:0/2534838397 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c0b2070 con 0x7f6a30004020 2024-03-18T15:34:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.860+0000 7f6a467fc640 1 -- 172.21.15.5:0/2534838397 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a3c0a5640 con 0x7f6a30004020 2024-03-18T15:34:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.860+0000 7f6a467fc640 1 -- 172.21.15.5:0/2534838397 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c009690 con 0x7f6a30004020 2024-03-18T15:34:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.860+0000 7f6a4deda640 1 -- 172.21.15.5:0/2534838397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a30004020 msgr2=0x7f6a30004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.860+0000 7f6a4deda640 1 --2- 172.21.15.5:0/2534838397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a30004020 0x7f6a30004810 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c009ac0 tx=0x7f6a3c01ca50 comp rx=0 tx=0).stop 2024-03-18T15:34:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.860+0000 7f6a47fff640 1 -- 172.21.15.5:0/2534838397 reap_dead start 2024-03-18T15:34:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.860+0000 7f6a4deda640 1 -- 172.21.15.5:0/2534838397 shutdown_connections 2024-03-18T15:34:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.860+0000 7f6a4deda640 1 -- 172.21.15.5:0/2534838397 >> 172.21.15.5:0/2534838397 conn(0x7f6a48066fc0 msgr2=0x7f6a480673d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.860+0000 7f6a4deda640 1 -- 172.21.15.5:0/2534838397 shutdown_connections 2024-03-18T15:34:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.860+0000 7f6a4deda640 1 -- 172.21.15.5:0/2534838397 wait complete. 2024-03-18T15:34:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.864+0000 7f6a4deda640 1 Processor -- start 2024-03-18T15:34:32.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.864+0000 7f6a4deda640 1 -- start start 2024-03-18T15:34:32.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.864+0000 7f6a4deda640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6a30004020 0x7f6a480b7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:32.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.864+0000 7f6a4deda640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6a481d2500 0x7f6a481d34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:32.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.864+0000 7f6a4deda640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a481d39e0 0x7f6a481494b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:32.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.864+0000 7f6a4deda640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6a480b4290 con 0x7f6a481d39e0 2024-03-18T15:34:32.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.864+0000 7f6a4deda640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f6a480b4110 con 0x7f6a30004020 2024-03-18T15:34:32.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.864+0000 7f6a4deda640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f6a481472a0 con 0x7f6a481d2500 2024-03-18T15:34:32.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.864+0000 7f6a47fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a481d39e0 0x7f6a481494b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:32.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.864+0000 7f6a47fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a481d39e0 0x7f6a481494b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56448/0 (socket says 172.21.15.5:56448) 2024-03-18T15:34:32.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.864+0000 7f6a47fff640 1 -- 172.21.15.5:0/2981328558 learned_addr learned my addr 172.21.15.5:0/2981328558 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:32.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.864+0000 7f6a477fe640 1 --2- 172.21.15.5:0/2981328558 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6a30004020 0x7f6a480b7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:32.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.864+0000 7f6a46ffd640 1 --2- 172.21.15.5:0/2981328558 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6a481d2500 0x7f6a481d34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:32.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.864+0000 7f6a47fff640 1 -- 172.21.15.5:0/2981328558 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6a481d2500 msgr2=0x7f6a481d34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:32.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.864+0000 7f6a47fff640 1 --2- 172.21.15.5:0/2981328558 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6a481d2500 0x7f6a481d34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:32.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.864+0000 7f6a47fff640 1 -- 172.21.15.5:0/2981328558 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6a30004020 msgr2=0x7f6a480b7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:32.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.864+0000 7f6a47fff640 1 --2- 172.21.15.5:0/2981328558 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6a30004020 0x7f6a480b7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:32.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.864+0000 7f6a47fff640 1 -- 172.21.15.5:0/2981328558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a481d2d30 con 0x7f6a481d39e0 2024-03-18T15:34:32.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.864+0000 7f6a47fff640 1 --2- 172.21.15.5:0/2981328558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a481d39e0 0x7f6a481494b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6a480b89c0 tx=0x7f6a40010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:32.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.864+0000 7f6a2ffff640 1 -- 172.21.15.5:0/2981328558 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4001a070 con 0x7f6a481d39e0 2024-03-18T15:34:32.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.864+0000 7f6a2ffff640 1 -- 172.21.15.5:0/2981328558 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a40011040 con 0x7f6a481d39e0 2024-03-18T15:34:32.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.864+0000 7f6a4deda640 1 -- 172.21.15.5:0/2981328558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a481d3010 con 0x7f6a481d39e0 2024-03-18T15:34:32.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.864+0000 7f6a2ffff640 1 -- 172.21.15.5:0/2981328558 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4000ed40 con 0x7f6a481d39e0 2024-03-18T15:34:32.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.864+0000 7f6a4deda640 1 -- 172.21.15.5:0/2981328558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a481d29b0 con 0x7f6a481d39e0 2024-03-18T15:34:32.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.864+0000 7f6a4deda640 1 -- 172.21.15.5:0/2981328558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a4814a890 con 0x7f6a481d39e0 2024-03-18T15:34:32.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.868+0000 7f6a2ffff640 1 -- 172.21.15.5:0/2981328558 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f6a4001d070 con 0x7f6a481d39e0 2024-03-18T15:34:32.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.868+0000 7f6a2ffff640 1 --2- 172.21.15.5:0/2981328558 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6a34067690 0x7f6a34069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:32.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.868+0000 7f6a2ffff640 1 -- 172.21.15.5:0/2981328558 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7f6a400886b0 con 0x7f6a481d39e0 2024-03-18T15:34:32.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.868+0000 7f6a477fe640 1 --2- 172.21.15.5:0/2981328558 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6a34067690 0x7f6a34069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:32.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.868+0000 7f6a477fe640 1 --2- 172.21.15.5:0/2981328558 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6a34067690 0x7f6a34069b50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6a480b8650 tx=0x7f6a38008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:32.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:32.872+0000 7f6a2ffff640 1 -- 172.21.15.5:0/2981328558 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f6a40058e50 con 0x7f6a481d39e0 2024-03-18T15:34:33.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.036+0000 7f6a4deda640 1 -- 172.21.15.5:0/2981328558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6a4804f550 con 0x7f6a481d39e0 2024-03-18T15:34:33.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.036+0000 7f6a2ffff640 1 -- 172.21.15.5:0/2981328558 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) ==== 100+0+11 (secure 0 0 0) 0x7f6a4005cef0 con 0x7f6a481d39e0 2024-03-18T15:34:33.039 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-03-18T15:34:33.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.036+0000 7f6a4deda640 1 -- 172.21.15.5:0/2981328558 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6a34067690 msgr2=0x7f6a34069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:33.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.036+0000 7f6a4deda640 1 --2- 172.21.15.5:0/2981328558 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6a34067690 0x7f6a34069b50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6a480b8650 tx=0x7f6a38008040 comp rx=0 tx=0).stop 2024-03-18T15:34:33.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.036+0000 7f6a4deda640 1 -- 172.21.15.5:0/2981328558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a481d39e0 msgr2=0x7f6a481494b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:33.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.036+0000 7f6a4deda640 1 --2- 172.21.15.5:0/2981328558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a481d39e0 0x7f6a481494b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6a480b89c0 tx=0x7f6a40010d70 comp rx=0 tx=0).stop 2024-03-18T15:34:33.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.036+0000 7f6a4deda640 1 -- 172.21.15.5:0/2981328558 shutdown_connections 2024-03-18T15:34:33.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.036+0000 7f6a4deda640 1 --2- 172.21.15.5:0/2981328558 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6a34067690 0x7f6a34069b50 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:33.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.036+0000 7f6a4deda640 1 --2- 172.21.15.5:0/2981328558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a481d39e0 0x7f6a481494b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:33.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.036+0000 7f6a4deda640 1 --2- 172.21.15.5:0/2981328558 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6a481d2500 0x7f6a481d34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:33.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.036+0000 7f6a4deda640 1 --2- 172.21.15.5:0/2981328558 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6a30004020 0x7f6a480b7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:33.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.036+0000 7f6a4deda640 1 -- 172.21.15.5:0/2981328558 >> 172.21.15.5:0/2981328558 conn(0x7f6a48066fc0 msgr2=0x7f6a480b6d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:33.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.036+0000 7f6a4deda640 1 -- 172.21.15.5:0/2981328558 shutdown_connections 2024-03-18T15:34:33.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.036+0000 7f6a4deda640 1 -- 172.21.15.5:0/2981328558 wait complete. 2024-03-18T15:34:33.063 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-18T15:34:33.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.188+0000 7f53b75b3640 1 Processor -- start 2024-03-18T15:34:33.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.188+0000 7f53b75b3640 1 -- start start 2024-03-18T15:34:33.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.188+0000 7f53b75b3640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f53b013dfa0 0x7f53b013e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:33.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.188+0000 7f53b75b3640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53b0148490 con 0x7f53b01479a0 2024-03-18T15:34:33.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.188+0000 7f53b75b3640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53b0072a70 con 0x7f53b0070d00 2024-03-18T15:34:33.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.188+0000 7f53b75b3640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f53b00702b0 con 0x7f53b013dfa0 2024-03-18T15:34:33.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.188+0000 7f53b534f640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f53b01479a0 0x7f53b0147d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44292/0 (socket says 172.21.15.5:44292) 2024-03-18T15:34:33.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.188+0000 7f53b534f640 1 -- 172.21.15.5:0/1098406332 learned_addr learned my addr 172.21.15.5:0/1098406332 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:33.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.188+0000 7f53a7fff640 1 -- 172.21.15.5:0/1098406332 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2464270800 0 0) 0x7f53b0148490 con 0x7f53b01479a0 2024-03-18T15:34:33.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.188+0000 7f53a7fff640 1 -- 172.21.15.5:0/1098406332 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f539c003530 con 0x7f53b01479a0 2024-03-18T15:34:33.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.188+0000 7f53b5b50640 1 --2- 172.21.15.5:0/1098406332 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f53b013dfa0 0x7f53b013e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:33.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.188+0000 7f53a7fff640 1 -- 172.21.15.5:0/1098406332 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 258485318 0 0) 0x7f539c003530 con 0x7f53b01479a0 2024-03-18T15:34:33.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.188+0000 7f53a7fff640 1 -- 172.21.15.5:0/1098406332 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f53b013dfa0 msgr2=0x7f53b013e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:33.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.188+0000 7f53a7fff640 1 --2- 172.21.15.5:0/1098406332 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f53b013dfa0 0x7f53b013e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:33.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.188+0000 7f53a7fff640 1 -- 172.21.15.5:0/1098406332 >> v1:172.21.15.80:6789/0 conn(0x7f53b0070d00 legacy=0x7f53b0071160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:33.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.188+0000 7f53a7fff640 1 -- 172.21.15.5:0/1098406332 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53b0070500 con 0x7f53b01479a0 2024-03-18T15:34:33.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.188+0000 7f53a7fff640 1 -- 172.21.15.5:0/1098406332 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3837549003 0 0) 0x7f53ac001680 con 0x7f53b01479a0 2024-03-18T15:34:33.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.188+0000 7f53a7fff640 1 -- 172.21.15.5:0/1098406332 >> v1:172.21.15.5:6789/0 conn(0x7f53b01479a0 legacy=0x7f53b0147d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:33.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.188+0000 7f53a7fff640 1 --2- 172.21.15.5:0/1098406332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f539c003ad0 0x7f539c003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:33.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.188+0000 7f53a7fff640 1 --2- 172.21.15.5:0/1098406332 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f53b013dfa0 0x7f539c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:33.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.188+0000 7f53a7fff640 1 --2- 172.21.15.5:0/1098406332 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f539c004dc0 0x7f539c005240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:33.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.188+0000 7f53a7fff640 1 -- 172.21.15.5:0/1098406332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f53b00702b0 con 0x7f539c003ad0 2024-03-18T15:34:33.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.188+0000 7f53a7fff640 1 -- 172.21.15.5:0/1098406332 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f539c00bca0 con 0x7f539c004dc0 2024-03-18T15:34:33.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.188+0000 7f53a7fff640 1 -- 172.21.15.5:0/1098406332 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f539c00bea0 con 0x7f53b013dfa0 2024-03-18T15:34:33.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.188+0000 7f53b534f640 1 --2- 172.21.15.5:0/1098406332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f539c003ad0 0x7f539c003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:33.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.188+0000 7f53b4b4e640 1 --2- 172.21.15.5:0/1098406332 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f53b013dfa0 0x7f539c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:33.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.188+0000 7f53b5b50640 1 --2- 172.21.15.5:0/1098406332 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f539c004dc0 0x7f539c005240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:33.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.188+0000 7f53b534f640 1 -- 172.21.15.5:0/1098406332 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f53b013dfa0 msgr2=0x7f539c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:33.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.188+0000 7f53b534f640 1 --2- 172.21.15.5:0/1098406332 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f53b013dfa0 0x7f539c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:33.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.188+0000 7f53b534f640 1 -- 172.21.15.5:0/1098406332 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f539c004dc0 msgr2=0x7f539c005240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:33.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.188+0000 7f53b534f640 1 --2- 172.21.15.5:0/1098406332 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f539c004dc0 0x7f539c005240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:33.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.188+0000 7f53b534f640 1 -- 172.21.15.5:0/1098406332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f539c003530 con 0x7f539c003ad0 2024-03-18T15:34:33.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.188+0000 7f53b5b50640 1 --2- 172.21.15.5:0/1098406332 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f539c004dc0 0x7f539c005240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:33.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.192+0000 7f53b534f640 1 --2- 172.21.15.5:0/1098406332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f539c003ad0 0x7f539c003ef0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f53ac00c240 tx=0x7f53ac01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b13f80e8bb57d1e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:33.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.192+0000 7f53a7fff640 1 -- 172.21.15.5:0/1098406332 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ac007d20 con 0x7f539c003ad0 2024-03-18T15:34:33.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.192+0000 7f53a7fff640 1 -- 172.21.15.5:0/1098406332 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53ac0a9070 con 0x7f539c003ad0 2024-03-18T15:34:33.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.192+0000 7f53a7fff640 1 -- 172.21.15.5:0/1098406332 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ac005d60 con 0x7f539c003ad0 2024-03-18T15:34:33.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.192+0000 7f53b75b3640 1 -- 172.21.15.5:0/1098406332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f539c003ad0 msgr2=0x7f539c003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:33.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.192+0000 7f53b75b3640 1 --2- 172.21.15.5:0/1098406332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f539c003ad0 0x7f539c003ef0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f53ac00c240 tx=0x7f53ac01cc80 comp rx=0 tx=0).stop 2024-03-18T15:34:33.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.192+0000 7f53b5b50640 1 -- 172.21.15.5:0/1098406332 reap_dead start 2024-03-18T15:34:33.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.192+0000 7f53b75b3640 1 -- 172.21.15.5:0/1098406332 shutdown_connections 2024-03-18T15:34:33.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.192+0000 7f53b75b3640 1 -- 172.21.15.5:0/1098406332 >> 172.21.15.5:0/1098406332 conn(0x7f53b00b7a80 msgr2=0x7f53b00b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:33.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.192+0000 7f53b75b3640 1 -- 172.21.15.5:0/1098406332 shutdown_connections 2024-03-18T15:34:33.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.192+0000 7f53b75b3640 1 -- 172.21.15.5:0/1098406332 wait complete. 2024-03-18T15:34:33.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.192+0000 7f53b75b3640 1 Processor -- start 2024-03-18T15:34:33.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.192+0000 7f53b75b3640 1 -- start start 2024-03-18T15:34:33.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.196+0000 7f53b75b3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f539c003ad0 0x7f53b00b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:33.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.196+0000 7f53b75b3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53b00b3870 0x7f53b01dfe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:33.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.196+0000 7f53b75b3640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f53b01e0340 0x7f53b01358a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:33.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.196+0000 7f53b75b3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f53b006be60 con 0x7f53b00b3870 2024-03-18T15:34:33.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.196+0000 7f53b75b3640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f53b006bce0 con 0x7f539c003ad0 2024-03-18T15:34:33.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.196+0000 7f53b75b3640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f53b006bfe0 con 0x7f53b01e0340 2024-03-18T15:34:33.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.196+0000 7f53b4b4e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53b00b3870 0x7f53b01dfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:33.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.196+0000 7f53b4b4e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53b00b3870 0x7f53b01dfe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56466/0 (socket says 172.21.15.5:56466) 2024-03-18T15:34:33.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.196+0000 7f53b4b4e640 1 -- 172.21.15.5:0/124220770 learned_addr learned my addr 172.21.15.5:0/124220770 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:33.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.196+0000 7f53b534f640 1 --2- 172.21.15.5:0/124220770 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f539c003ad0 0x7f53b00b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:33.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.196+0000 7f53b5b50640 1 --2- 172.21.15.5:0/124220770 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f53b01e0340 0x7f53b01358a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:33.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.196+0000 7f53b4b4e640 1 -- 172.21.15.5:0/124220770 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f53b01e0340 msgr2=0x7f53b01358a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:33.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.196+0000 7f53b4b4e640 1 --2- 172.21.15.5:0/124220770 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f53b01e0340 0x7f53b01358a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:33.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.196+0000 7f53b4b4e640 1 -- 172.21.15.5:0/124220770 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f539c003ad0 msgr2=0x7f53b00b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:33.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.196+0000 7f53b4b4e640 1 --2- 172.21.15.5:0/124220770 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f539c003ad0 0x7f53b00b2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:33.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.196+0000 7f53b4b4e640 1 -- 172.21.15.5:0/124220770 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53b00b5bb0 con 0x7f53b00b3870 2024-03-18T15:34:33.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.196+0000 7f53b4b4e640 1 --2- 172.21.15.5:0/124220770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53b00b3870 0x7f53b01dfe00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f53a000e500 tx=0x7f53a000e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:33.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.196+0000 7f53a5ffb640 1 -- 172.21.15.5:0/124220770 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a0019070 con 0x7f53b00b3870 2024-03-18T15:34:33.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.196+0000 7f53a5ffb640 1 -- 172.21.15.5:0/124220770 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53a00052e0 con 0x7f53b00b3870 2024-03-18T15:34:33.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.196+0000 7f53b75b3640 1 -- 172.21.15.5:0/124220770 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53b01e07c0 con 0x7f53b00b3870 2024-03-18T15:34:33.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.196+0000 7f53a5ffb640 1 -- 172.21.15.5:0/124220770 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a0003840 con 0x7f53b00b3870 2024-03-18T15:34:33.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.196+0000 7f53b75b3640 1 -- 172.21.15.5:0/124220770 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53b0142550 con 0x7f53b00b3870 2024-03-18T15:34:33.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.196+0000 7f53b75b3640 1 -- 172.21.15.5:0/124220770 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53b004f550 con 0x7f53b00b3870 2024-03-18T15:34:33.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.196+0000 7f53a5ffb640 1 -- 172.21.15.5:0/124220770 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f53a001c070 con 0x7f53b00b3870 2024-03-18T15:34:33.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.196+0000 7f53a5ffb640 1 --2- 172.21.15.5:0/124220770 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5394067660 0x7f5394069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:33.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.196+0000 7f53a5ffb640 1 -- 172.21.15.5:0/124220770 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7f53a0085560 con 0x7f53b00b3870 2024-03-18T15:34:33.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.196+0000 7f53b534f640 1 --2- 172.21.15.5:0/124220770 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5394067660 0x7f5394069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:33.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.200+0000 7f53b534f640 1 --2- 172.21.15.5:0/124220770 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5394067660 0x7f5394069b20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f53ac0ab000 tx=0x7f53ac0b1040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:33.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.200+0000 7f53a5ffb640 1 -- 172.21.15.5:0/124220770 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f53b01e07c0 con 0x7f53b00b3870 2024-03-18T15:34:33.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.372+0000 7f53b75b3640 1 -- 172.21.15.5:0/124220770 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f53b0142830 con 0x7f53b00b3870 2024-03-18T15:34:33.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.372+0000 7f53a5ffb640 1 -- 172.21.15.5:0/124220770 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v30) ==== 80+0+0 (secure 0 0 0) 0x7f53a0055c20 con 0x7f53b00b3870 2024-03-18T15:34:33.377 INFO:teuthology.orchestra.run.smithi005.stderr:listed 0 entries 2024-03-18T15:34:33.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.376+0000 7f53b75b3640 1 -- 172.21.15.5:0/124220770 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5394067660 msgr2=0x7f5394069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:33.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.376+0000 7f53b75b3640 1 --2- 172.21.15.5:0/124220770 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5394067660 0x7f5394069b20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f53ac0ab000 tx=0x7f53ac0b1040 comp rx=0 tx=0).stop 2024-03-18T15:34:33.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.376+0000 7f53b75b3640 1 -- 172.21.15.5:0/124220770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53b00b3870 msgr2=0x7f53b01dfe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:33.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.376+0000 7f53b75b3640 1 --2- 172.21.15.5:0/124220770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53b00b3870 0x7f53b01dfe00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f53a000e500 tx=0x7f53a000e9c0 comp rx=0 tx=0).stop 2024-03-18T15:34:33.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.376+0000 7f53b75b3640 1 -- 172.21.15.5:0/124220770 shutdown_connections 2024-03-18T15:34:33.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.376+0000 7f53b75b3640 1 --2- 172.21.15.5:0/124220770 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5394067660 0x7f5394069b20 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:33.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.376+0000 7f53b75b3640 1 --2- 172.21.15.5:0/124220770 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f53b01e0340 0x7f53b01358a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:33.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.376+0000 7f53b75b3640 1 --2- 172.21.15.5:0/124220770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53b00b3870 0x7f53b01dfe00 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:33.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.376+0000 7f53b75b3640 1 --2- 172.21.15.5:0/124220770 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f539c003ad0 0x7f53b00b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:33.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.376+0000 7f53b75b3640 1 -- 172.21.15.5:0/124220770 >> 172.21.15.5:0/124220770 conn(0x7f53b00b7a80 msgr2=0x7f53b014cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:33.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.376+0000 7f53b75b3640 1 -- 172.21.15.5:0/124220770 shutdown_connections 2024-03-18T15:34:33.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.376+0000 7f53b75b3640 1 -- 172.21.15.5:0/124220770 wait complete. 2024-03-18T15:34:33.395 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi005.front.sepia.ceph.com... 2024-03-18T15:34:33.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:34:33.395 DEBUG:teuthology.orchestra.run.smithi005:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-18T15:34:33.404 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi005.stderr:ceph-fuse[35096]: fuse finished with error 0 and tester_r 0 2024-03-18T15:34:33.405 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-03-18T15:34:33.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:34:33.406 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-18T15:34:33.460 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:34:33.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.616+0000 7f6548d1b640 1 Processor -- start 2024-03-18T15:34:33.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.616+0000 7f6548d1b640 1 -- start start 2024-03-18T15:34:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.616+0000 7f6548d1b640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f65441417e0 0x7f6544070360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.616+0000 7f6548d1b640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f654406b8c0 con 0x7f654414b660 2024-03-18T15:34:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.616+0000 7f6548d1b640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f654414ab80 con 0x7f654406b4e0 2024-03-18T15:34:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.616+0000 7f6548d1b640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f654414b1d0 con 0x7f65441417e0 2024-03-18T15:34:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.616+0000 7f6541d9b640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f654414b660 0x7f65441410d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44296/0 (socket says 172.21.15.5:44296) 2024-03-18T15:34:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.616+0000 7f6541d9b640 1 -- 172.21.15.5:0/84207775 learned_addr learned my addr 172.21.15.5:0/84207775 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.616+0000 7f6542d9d640 1 --2- 172.21.15.5:0/84207775 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f65441417e0 0x7f6544070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:33.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.616+0000 7f654159a640 1 -- 172.21.15.5:0/84207775 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4192631092 0 0) 0x7f654406b8c0 con 0x7f654414b660 2024-03-18T15:34:33.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.616+0000 7f654159a640 1 -- 172.21.15.5:0/84207775 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6530003530 con 0x7f654414b660 2024-03-18T15:34:33.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.616+0000 7f654159a640 1 -- 172.21.15.5:0/84207775 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1889714726 0 0) 0x7f6530003530 con 0x7f654414b660 2024-03-18T15:34:33.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.616+0000 7f654159a640 1 -- 172.21.15.5:0/84207775 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f65441417e0 msgr2=0x7f6544070360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:33.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.616+0000 7f654159a640 1 --2- 172.21.15.5:0/84207775 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f65441417e0 0x7f6544070360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:33.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.616+0000 7f654159a640 1 -- 172.21.15.5:0/84207775 >> v1:172.21.15.80:6789/0 conn(0x7f654406b4e0 legacy=0x7f6544071da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:33.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.616+0000 7f654159a640 1 -- 172.21.15.5:0/84207775 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f654414ae60 con 0x7f654414b660 2024-03-18T15:34:33.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.616+0000 7f654159a640 1 -- 172.21.15.5:0/84207775 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3837549003 0 0) 0x7f652c001680 con 0x7f654414b660 2024-03-18T15:34:33.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.616+0000 7f654159a640 1 -- 172.21.15.5:0/84207775 >> v1:172.21.15.5:6789/0 conn(0x7f654414b660 legacy=0x7f65441410d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:33.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.616+0000 7f654159a640 1 --2- 172.21.15.5:0/84207775 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f65441417e0 0x7f6530003af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:33.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.616+0000 7f654159a640 1 --2- 172.21.15.5:0/84207775 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6530004070 0x7f6530004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:33.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.616+0000 7f654159a640 1 --2- 172.21.15.5:0/84207775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6530004d50 0x7f65300051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:33.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.616+0000 7f654159a640 1 -- 172.21.15.5:0/84207775 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f654414b1d0 con 0x7f6530004d50 2024-03-18T15:34:33.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.616+0000 7f654159a640 1 -- 172.21.15.5:0/84207775 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f653000bcb0 con 0x7f6530004070 2024-03-18T15:34:33.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.616+0000 7f654159a640 1 -- 172.21.15.5:0/84207775 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f653000beb0 con 0x7f65441417e0 2024-03-18T15:34:33.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.616+0000 7f6542d9d640 1 --2- 172.21.15.5:0/84207775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6530004d50 0x7f65300051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:33.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.616+0000 7f6541d9b640 1 --2- 172.21.15.5:0/84207775 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6530004070 0x7f6530004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:33.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.616+0000 7f654259c640 1 --2- 172.21.15.5:0/84207775 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f65441417e0 0x7f6530003af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:33.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.616+0000 7f6542d9d640 1 -- 172.21.15.5:0/84207775 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f65441417e0 msgr2=0x7f6530003af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:33.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.616+0000 7f6542d9d640 1 --2- 172.21.15.5:0/84207775 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f65441417e0 0x7f6530003af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:33.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.616+0000 7f6542d9d640 1 -- 172.21.15.5:0/84207775 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6530004070 msgr2=0x7f6530004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:33.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.616+0000 7f6542d9d640 1 --2- 172.21.15.5:0/84207775 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6530004070 0x7f6530004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:33.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.616+0000 7f6542d9d640 1 -- 172.21.15.5:0/84207775 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6530003530 con 0x7f6530004d50 2024-03-18T15:34:33.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.620+0000 7f6542d9d640 1 --2- 172.21.15.5:0/84207775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6530004d50 0x7f65300051d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f653c009ac0 tx=0x7f653c01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3184b29637bf1773 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:33.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.620+0000 7f654159a640 1 -- 172.21.15.5:0/84207775 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653c0b2070 con 0x7f6530004d50 2024-03-18T15:34:33.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.620+0000 7f654159a640 1 -- 172.21.15.5:0/84207775 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f653c0a5530 con 0x7f6530004d50 2024-03-18T15:34:33.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.620+0000 7f654159a640 1 -- 172.21.15.5:0/84207775 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653c009690 con 0x7f6530004d50 2024-03-18T15:34:33.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.620+0000 7f6548d1b640 1 -- 172.21.15.5:0/84207775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6530004d50 msgr2=0x7f65300051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:33.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.620+0000 7f6548d1b640 1 --2- 172.21.15.5:0/84207775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6530004d50 0x7f65300051d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f653c009ac0 tx=0x7f653c01ca50 comp rx=0 tx=0).stop 2024-03-18T15:34:33.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.620+0000 7f6542d9d640 1 -- 172.21.15.5:0/84207775 reap_dead start 2024-03-18T15:34:33.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.620+0000 7f6548d1b640 1 -- 172.21.15.5:0/84207775 shutdown_connections 2024-03-18T15:34:33.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.620+0000 7f6548d1b640 1 -- 172.21.15.5:0/84207775 >> 172.21.15.5:0/84207775 conn(0x7f65440b6680 msgr2=0x7f65440b6a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:33.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.620+0000 7f6548d1b640 1 -- 172.21.15.5:0/84207775 shutdown_connections 2024-03-18T15:34:33.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.620+0000 7f6548d1b640 1 -- 172.21.15.5:0/84207775 wait complete. 2024-03-18T15:34:33.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.620+0000 7f6548d1b640 1 Processor -- start 2024-03-18T15:34:33.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.620+0000 7f6548d1b640 1 -- start start 2024-03-18T15:34:33.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.620+0000 7f6548d1b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65440af960 0x7f65441dcff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:33.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.620+0000 7f6548d1b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6544141da0 0x7f65441dd530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:33.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.620+0000 7f6548d1b640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6544139cc0 0x7f654413a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:33.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.620+0000 7f6548d1b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6544070360 con 0x7f65440af960 2024-03-18T15:34:33.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.620+0000 7f6548d1b640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f6544071630 con 0x7f6544141da0 2024-03-18T15:34:33.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.620+0000 7f6548d1b640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f65440704e0 con 0x7f6544139cc0 2024-03-18T15:34:33.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.620+0000 7f654259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65440af960 0x7f65441dcff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:33.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.620+0000 7f654259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65440af960 0x7f65441dcff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56482/0 (socket says 172.21.15.5:56482) 2024-03-18T15:34:33.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.620+0000 7f654259c640 1 -- 172.21.15.5:0/867902609 learned_addr learned my addr 172.21.15.5:0/867902609 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:33.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.620+0000 7f6541d9b640 1 --2- 172.21.15.5:0/867902609 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6544141da0 0x7f65441dd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:33.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.620+0000 7f6542d9d640 1 --2- 172.21.15.5:0/867902609 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6544139cc0 0x7f654413a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:33.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.620+0000 7f654259c640 1 -- 172.21.15.5:0/867902609 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6544139cc0 msgr2=0x7f654413a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:33.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.624+0000 7f654259c640 1 --2- 172.21.15.5:0/867902609 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6544139cc0 0x7f654413a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:33.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.624+0000 7f654259c640 1 -- 172.21.15.5:0/867902609 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6544141da0 msgr2=0x7f65441dd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:33.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.624+0000 7f654259c640 1 --2- 172.21.15.5:0/867902609 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6544141da0 0x7f65441dd530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:33.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.624+0000 7f654259c640 1 -- 172.21.15.5:0/867902609 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6544142360 con 0x7f65440af960 2024-03-18T15:34:33.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.624+0000 7f6541d9b640 1 --2- 172.21.15.5:0/867902609 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6544141da0 0x7f65441dd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:34:33.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.624+0000 7f654259c640 1 --2- 172.21.15.5:0/867902609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65440af960 0x7f65441dcff0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f653400d930 tx=0x7f653400ddf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:33.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.624+0000 7f653affd640 1 -- 172.21.15.5:0/867902609 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6534003db0 con 0x7f65440af960 2024-03-18T15:34:33.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.624+0000 7f653affd640 1 -- 172.21.15.5:0/867902609 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6534017070 con 0x7f65440af960 2024-03-18T15:34:33.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.624+0000 7f653affd640 1 -- 172.21.15.5:0/867902609 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653400b970 con 0x7f65440af960 2024-03-18T15:34:33.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.624+0000 7f6548d1b640 1 -- 172.21.15.5:0/867902609 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6544142640 con 0x7f65440af960 2024-03-18T15:34:33.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.624+0000 7f6548d1b640 1 -- 172.21.15.5:0/867902609 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65440afd40 con 0x7f65440af960 2024-03-18T15:34:33.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.624+0000 7f6548d1b640 1 -- 172.21.15.5:0/867902609 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65440b08c0 con 0x7f65440af960 2024-03-18T15:34:33.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.624+0000 7f653affd640 1 -- 172.21.15.5:0/867902609 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f653400bb10 con 0x7f65440af960 2024-03-18T15:34:33.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.624+0000 7f653affd640 1 --2- 172.21.15.5:0/867902609 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6524067690 0x7f6524069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:33.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.624+0000 7f6541d9b640 1 --2- 172.21.15.5:0/867902609 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6524067690 0x7f6524069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:33.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.624+0000 7f653affd640 1 -- 172.21.15.5:0/867902609 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7f6534085a90 con 0x7f65440af960 2024-03-18T15:34:33.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.624+0000 7f6541d9b640 1 --2- 172.21.15.5:0/867902609 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6524067690 0x7f6524069b50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f652c001cd0 tx=0x7f652c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:33.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.628+0000 7f653affd640 1 -- 172.21.15.5:0/867902609 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f65440b08c0 con 0x7f65440af960 2024-03-18T15:34:33.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.884+0000 7f6548d1b640 1 -- 172.21.15.5:0/867902609 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f65440b89f0 con 0x7f65440af960 2024-03-18T15:34:33.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.884+0000 7f653affd640 1 -- 172.21.15.5:0/867902609 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+3528 (secure 0 0 0) 0x7f65340567a0 con 0x7f65440af960 2024-03-18T15:34:33.888 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:34:33.888 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4619,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/2921781388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":2921781388},{"type":"v1","addr":"172.21.15.80:6833","nonce":2921781388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-18T15:34:22.108585+0000","modified":"2024-03-18T15:34:25.110609+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4968},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4968":{"gid":4968,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6835/2235154713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2235154713},{"type":"v1","addr":"172.21.15.5:6835","nonce":2235154713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4968,"qdb_cluster":[4968]},"id":2}]} 2024-03-18T15:34:33.888 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 16 2024-03-18T15:34:33.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.888+0000 7f6548d1b640 1 -- 172.21.15.5:0/867902609 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6524067690 msgr2=0x7f6524069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:33.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.888+0000 7f6548d1b640 1 --2- 172.21.15.5:0/867902609 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6524067690 0x7f6524069b50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f652c001cd0 tx=0x7f652c009040 comp rx=0 tx=0).stop 2024-03-18T15:34:33.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.888+0000 7f6548d1b640 1 -- 172.21.15.5:0/867902609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65440af960 msgr2=0x7f65441dcff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:33.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.888+0000 7f6548d1b640 1 --2- 172.21.15.5:0/867902609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65440af960 0x7f65441dcff0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f653400d930 tx=0x7f653400ddf0 comp rx=0 tx=0).stop 2024-03-18T15:34:33.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.888+0000 7f6548d1b640 1 -- 172.21.15.5:0/867902609 shutdown_connections 2024-03-18T15:34:33.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.888+0000 7f6548d1b640 1 --2- 172.21.15.5:0/867902609 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6524067690 0x7f6524069b50 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:33.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.888+0000 7f6548d1b640 1 --2- 172.21.15.5:0/867902609 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6544139cc0 0x7f654413a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:33.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.888+0000 7f6548d1b640 1 --2- 172.21.15.5:0/867902609 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6544141da0 0x7f65441dd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:33.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.888+0000 7f6548d1b640 1 --2- 172.21.15.5:0/867902609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65440af960 0x7f65441dcff0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:33.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.888+0000 7f6548d1b640 1 -- 172.21.15.5:0/867902609 >> 172.21.15.5:0/867902609 conn(0x7f65440b6680 msgr2=0x7f6544140ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:33.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.888+0000 7f6548d1b640 1 -- 172.21.15.5:0/867902609 shutdown_connections 2024-03-18T15:34:33.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:33.888+0000 7f6548d1b640 1 -- 172.21.15.5:0/867902609 wait complete. 2024-03-18T15:34:33.907 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:34:34.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.024+0000 7f6e78074640 1 Processor -- start 2024-03-18T15:34:34.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.024+0000 7f6e78074640 1 -- start start 2024-03-18T15:34:34.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.028+0000 7f6e78074640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6e700744b0 0x7f6e700716f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:34.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.028+0000 7f6e78074640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f6e70145da0 0x7f6e700b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:34.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.028+0000 7f6e78074640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6e700b5290 0x7f6e70148890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:34.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.028+0000 7f6e78074640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f6e70145910 con 0x7f6e700744b0 2024-03-18T15:34:34.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.028+0000 7f6e78074640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f6e70145320 con 0x7f6e70145da0 2024-03-18T15:34:34.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.028+0000 7f6e78074640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f6e70074890 con 0x7f6e700b5290 2024-03-18T15:34:34.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.028+0000 7f6e75e10640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6e700744b0 0x7f6e700716f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:34.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.028+0000 7f6e75e10640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f6e700744b0 0x7f6e700716f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56486/0 (socket says 172.21.15.5:56486) 2024-03-18T15:34:34.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.028+0000 7f6e75e10640 1 -- 172.21.15.5:0/569099220 learned_addr learned my addr 172.21.15.5:0/569099220 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:34.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.028+0000 7f6e76611640 1 --2- 172.21.15.5:0/569099220 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6e700b5290 0x7f6e70148890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:34.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.028+0000 7f6e7560f640 1 --2- 172.21.15.5:0/569099220 >> v2:172.21.15.80:3300/0 conn(0x7f6e70145da0 0x7f6e700b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:34.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.028+0000 7f6e75e10640 1 -- 172.21.15.5:0/569099220 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6e700b5290 msgr2=0x7f6e70148890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:34.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.028+0000 7f6e75e10640 1 --2- 172.21.15.5:0/569099220 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6e700b5290 0x7f6e70148890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:34.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.028+0000 7f6e75e10640 1 -- 172.21.15.5:0/569099220 >> v2:172.21.15.80:3300/0 conn(0x7f6e70145da0 msgr2=0x7f6e700b4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:34.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.028+0000 7f6e75e10640 1 --2- 172.21.15.5:0/569099220 >> v2:172.21.15.80:3300/0 conn(0x7f6e70145da0 0x7f6e700b4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:34.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.028+0000 7f6e75e10640 1 -- 172.21.15.5:0/569099220 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e70070f80 con 0x7f6e700744b0 2024-03-18T15:34:34.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.028+0000 7f6e75e10640 1 --2- 172.21.15.5:0/569099220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e700744b0 0x7f6e700716f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6e70149950 tx=0x7f6e6c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcbb8c3106f76e8a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:34.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.028+0000 7f6e74e0e640 1 -- 172.21.15.5:0/569099220 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c0b1070 con 0x7f6e700744b0 2024-03-18T15:34:34.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.028+0000 7f6e74e0e640 1 -- 172.21.15.5:0/569099220 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e6c0a8050 con 0x7f6e700744b0 2024-03-18T15:34:34.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.028+0000 7f6e74e0e640 1 -- 172.21.15.5:0/569099220 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c0ac860 con 0x7f6e700744b0 2024-03-18T15:34:34.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.032+0000 7f6e78074640 1 -- 172.21.15.5:0/569099220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e700744b0 msgr2=0x7f6e700716f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:34.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.032+0000 7f6e78074640 1 --2- 172.21.15.5:0/569099220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e700744b0 0x7f6e700716f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6e70149950 tx=0x7f6e6c01c460 comp rx=0 tx=0).stop 2024-03-18T15:34:34.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.032+0000 7f6e78074640 1 -- 172.21.15.5:0/569099220 shutdown_connections 2024-03-18T15:34:34.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.032+0000 7f6e78074640 1 --2- 172.21.15.5:0/569099220 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6e700b5290 0x7f6e70148890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:34.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.032+0000 7f6e78074640 1 --2- 172.21.15.5:0/569099220 >> v2:172.21.15.80:3300/0 conn(0x7f6e70145da0 0x7f6e700b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:34.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.032+0000 7f6e78074640 1 --2- 172.21.15.5:0/569099220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e700744b0 0x7f6e700716f0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:34.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.032+0000 7f6e78074640 1 -- 172.21.15.5:0/569099220 >> 172.21.15.5:0/569099220 conn(0x7f6e700b9190 msgr2=0x7f6e700b95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:34.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.032+0000 7f6e78074640 1 -- 172.21.15.5:0/569099220 shutdown_connections 2024-03-18T15:34:34.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.032+0000 7f6e78074640 1 -- 172.21.15.5:0/569099220 wait complete. 2024-03-18T15:34:34.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.032+0000 7f6e78074640 1 Processor -- start 2024-03-18T15:34:34.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.032+0000 7f6e78074640 1 -- start start 2024-03-18T15:34:34.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.032+0000 7f6e78074640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6e700744b0 0x7f6e700b87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:34.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.032+0000 7f6e78074640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6e700b5290 0x7f6e700b17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:34.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.032+0000 7f6e78074640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e70145da0 0x7f6e701e34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:34.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.032+0000 7f6e78074640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6e70070c00 con 0x7f6e70145da0 2024-03-18T15:34:34.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.032+0000 7f6e78074640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f6e700708e0 con 0x7f6e700744b0 2024-03-18T15:34:34.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.032+0000 7f6e78074640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f6e70146180 con 0x7f6e700b5290 2024-03-18T15:34:34.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.032+0000 7f6e76611640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e70145da0 0x7f6e701e34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:34.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.032+0000 7f6e76611640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e70145da0 0x7f6e701e34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56490/0 (socket says 172.21.15.5:56490) 2024-03-18T15:34:34.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.032+0000 7f6e76611640 1 -- 172.21.15.5:0/4199211366 learned_addr learned my addr 172.21.15.5:0/4199211366 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:34.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.032+0000 7f6e75e10640 1 --2- 172.21.15.5:0/4199211366 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6e700744b0 0x7f6e700b87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:34.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.032+0000 7f6e7560f640 1 --2- 172.21.15.5:0/4199211366 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6e700b5290 0x7f6e700b17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:34.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.032+0000 7f6e76611640 1 -- 172.21.15.5:0/4199211366 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6e700b5290 msgr2=0x7f6e700b17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:34.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.032+0000 7f6e76611640 1 --2- 172.21.15.5:0/4199211366 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6e700b5290 0x7f6e700b17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:34.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.032+0000 7f6e76611640 1 -- 172.21.15.5:0/4199211366 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6e700744b0 msgr2=0x7f6e700b87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:34.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.032+0000 7f6e76611640 1 --2- 172.21.15.5:0/4199211366 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6e700744b0 0x7f6e700b87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:34.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.032+0000 7f6e76611640 1 -- 172.21.15.5:0/4199211366 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e700b1d10 con 0x7f6e70145da0 2024-03-18T15:34:34.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.032+0000 7f6e7560f640 1 --2- 172.21.15.5:0/4199211366 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6e700b5290 0x7f6e700b17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:34.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.032+0000 7f6e76611640 1 --2- 172.21.15.5:0/4199211366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e70145da0 0x7f6e701e34f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6e68007e40 tx=0x7f6e6800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:34.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.032+0000 7f6e627fc640 1 -- 172.21.15.5:0/4199211366 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e68014620 con 0x7f6e70145da0 2024-03-18T15:34:34.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.032+0000 7f6e627fc640 1 -- 172.21.15.5:0/4199211366 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e68019070 con 0x7f6e70145da0 2024-03-18T15:34:34.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.032+0000 7f6e78074640 1 -- 172.21.15.5:0/4199211366 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e700b1f90 con 0x7f6e70145da0 2024-03-18T15:34:34.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.032+0000 7f6e627fc640 1 -- 172.21.15.5:0/4199211366 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e68014a20 con 0x7f6e70145da0 2024-03-18T15:34:34.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.032+0000 7f6e78074640 1 -- 172.21.15.5:0/4199211366 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e700b0d30 con 0x7f6e70145da0 2024-03-18T15:34:34.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.036+0000 7f6e78074640 1 -- 172.21.15.5:0/4199211366 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e7004f550 con 0x7f6e70145da0 2024-03-18T15:34:34.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.036+0000 7f6e627fc640 1 -- 172.21.15.5:0/4199211366 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f6e68014bc0 con 0x7f6e70145da0 2024-03-18T15:34:34.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.036+0000 7f6e627fc640 1 --2- 172.21.15.5:0/4199211366 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6e54067660 0x7f6e54069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:34.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.036+0000 7f6e627fc640 1 -- 172.21.15.5:0/4199211366 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7f6e680860f0 con 0x7f6e70145da0 2024-03-18T15:34:34.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.036+0000 7f6e75e10640 1 --2- 172.21.15.5:0/4199211366 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6e54067660 0x7f6e54069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:34.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.036+0000 7f6e75e10640 1 --2- 172.21.15.5:0/4199211366 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6e54067660 0x7f6e54069b20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c0ae000 tx=0x7f6e6c0a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:34.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.040+0000 7f6e627fc640 1 -- 172.21.15.5:0/4199211366 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f6e700b1d10 con 0x7f6e70145da0 2024-03-18T15:34:34.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.236+0000 7f6e78074640 1 -- 172.21.15.5:0/4199211366 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6e70143e80 con 0x7f6e70145da0 2024-03-18T15:34:34.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.236+0000 7f6e627fc640 1 -- 172.21.15.5:0/4199211366 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) ==== 74+0+16513 (secure 0 0 0) 0x7f6e68056e00 con 0x7f6e70145da0 2024-03-18T15:34:34.241 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:34:34.241 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":30,"fsid":"ef54fe03-5aa7-4eb2-9b1e-66c9cc345b94","created":"2024-03-18T15:33:18.086614+0000","modified":"2024-03-18T15:34:23.736672+0000","last_up_change":"2024-03-18T15:33:29.229911+0000","last_in_change":"2024-03-18T15:33:20.323840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:28.963186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-18T15:34:20.652674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-18T15:34:21.110594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df1c754b-af59-4d16-8919-fbb6386b3c48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6804","nonce":976264754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6808","nonce":976264754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6815","nonce":976264754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6811","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6813","nonce":976264754}]},"public_addr":"172.21.15.5:6804/976264754","cluster_addr":"172.21.15.5:6808/976264754","heartbeat_back_addr":"172.21.15.5:6815/976264754","heartbeat_front_addr":"172.21.15.5:6813/976264754","state":["exists","up"]},{"osd":1,"uuid":"b9a08b5e-8af4-4144-bdba-d69ed1ac627a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6801","nonce":3901761126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6803","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6805","nonce":3901761126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6812","nonce":3901761126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6807","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6809","nonce":3901761126}]},"public_addr":"172.21.15.5:6801/3901761126","cluster_addr":"172.21.15.5:6805/3901761126","heartbeat_back_addr":"172.21.15.5:6812/3901761126","heartbeat_front_addr":"172.21.15.5:6809/3901761126","state":["exists","up"]},{"osd":2,"uuid":"c3e8bb76-e382-439d-b380-62ef92f1255a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6817","nonce":502968448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6819","nonce":502968448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6823","nonce":502968448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6821","nonce":502968448}]},"public_addr":"172.21.15.5:6817/502968448","cluster_addr":"172.21.15.5:6819/502968448","heartbeat_back_addr":"172.21.15.5:6823/502968448","heartbeat_front_addr":"172.21.15.5:6821/502968448","state":["exists","up"]},{"osd":3,"uuid":"e8c94625-f2cd-4eba-b53d-2e7356e1c02a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6825","nonce":2425932735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6827","nonce":2425932735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6831","nonce":2425932735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6829","nonce":2425932735}]},"public_addr":"172.21.15.5:6825/2425932735","cluster_addr":"172.21.15.5:6827/2425932735","heartbeat_back_addr":"172.21.15.5:6831/2425932735","heartbeat_front_addr":"172.21.15.5:6829/2425932735","state":["exists","up"]},{"osd":4,"uuid":"0099dd52-6265-4d32-82e4-3ef16e73add0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6817","nonce":976514285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6819","nonce":976514285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6823","nonce":976514285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6821","nonce":976514285}]},"public_addr":"172.21.15.80:6817/976514285","cluster_addr":"172.21.15.80:6819/976514285","heartbeat_back_addr":"172.21.15.80:6823/976514285","heartbeat_front_addr":"172.21.15.80:6821/976514285","state":["exists","up"]},{"osd":5,"uuid":"ed10c84f-8c39-41cc-9b64-7c482a94a48e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6825","nonce":627153447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6827","nonce":627153447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6831","nonce":627153447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6829","nonce":627153447}]},"public_addr":"172.21.15.80:6825/627153447","cluster_addr":"172.21.15.80:6827/627153447","heartbeat_back_addr":"172.21.15.80:6831/627153447","heartbeat_front_addr":"172.21.15.80:6829/627153447","state":["exists","up"]},{"osd":6,"uuid":"c7a3538a-7048-4fba-8781-b440f03c25db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6803","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6805","nonce":488342527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6807","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6809","nonce":488342527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6815","nonce":488342527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6811","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6813","nonce":488342527}]},"public_addr":"172.21.15.80:6805/488342527","cluster_addr":"172.21.15.80:6809/488342527","heartbeat_back_addr":"172.21.15.80:6815/488342527","heartbeat_front_addr":"172.21.15.80:6813/488342527","state":["exists","up"]},{"osd":7,"uuid":"f9418104-3a97-4110-bc3a-1f8151d0f757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6801","nonce":982831666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6804","nonce":982831666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6812","nonce":982831666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6808","nonce":982831666}]},"public_addr":"172.21.15.80:6801/982831666","cluster_addr":"172.21.15.80:6804/982831666","heartbeat_back_addr":"172.21.15.80:6812/982831666","heartbeat_front_addr":"172.21.15.80:6808/982831666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.970828+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.858116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.987556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.864603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.756293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.591502+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:34:34.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.240+0000 7f6e78074640 1 -- 172.21.15.5:0/4199211366 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6e54067660 msgr2=0x7f6e54069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:34.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.240+0000 7f6e78074640 1 --2- 172.21.15.5:0/4199211366 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6e54067660 0x7f6e54069b20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c0ae000 tx=0x7f6e6c0a7000 comp rx=0 tx=0).stop 2024-03-18T15:34:34.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.240+0000 7f6e78074640 1 -- 172.21.15.5:0/4199211366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e70145da0 msgr2=0x7f6e701e34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:34.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.240+0000 7f6e78074640 1 --2- 172.21.15.5:0/4199211366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e70145da0 0x7f6e701e34f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6e68007e40 tx=0x7f6e6800e4b0 comp rx=0 tx=0).stop 2024-03-18T15:34:34.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.240+0000 7f6e78074640 1 -- 172.21.15.5:0/4199211366 shutdown_connections 2024-03-18T15:34:34.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.240+0000 7f6e78074640 1 --2- 172.21.15.5:0/4199211366 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6e54067660 0x7f6e54069b20 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:34.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.240+0000 7f6e78074640 1 --2- 172.21.15.5:0/4199211366 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e70145da0 0x7f6e701e34f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:34.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.240+0000 7f6e78074640 1 --2- 172.21.15.5:0/4199211366 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f6e700b5290 0x7f6e700b17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:34.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.240+0000 7f6e78074640 1 --2- 172.21.15.5:0/4199211366 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6e700744b0 0x7f6e700b87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:34.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.240+0000 7f6e78074640 1 -- 172.21.15.5:0/4199211366 >> 172.21.15.5:0/4199211366 conn(0x7f6e700b9190 msgr2=0x7f6e70143300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:34.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.240+0000 7f6e78074640 1 -- 172.21.15.5:0/4199211366 shutdown_connections 2024-03-18T15:34:34.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.240+0000 7f6e78074640 1 -- 172.21.15.5:0/4199211366 wait complete. 2024-03-18T15:34:34.266 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-18T15:33:28.963186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-18T15:34:20.652674+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '29', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-03-18T15:34:21.110594+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '30', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-18T15:34:34.266 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-18T15:34:34.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.396+0000 7f57ea20b640 1 Processor -- start 2024-03-18T15:34:34.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.396+0000 7f57ea20b640 1 -- start start 2024-03-18T15:34:34.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.396+0000 7f57ea20b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f57e40744d0 0x7f57e41455b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:34.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.396+0000 7f57ea20b640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f57e40b3050 0x7f57e4145af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:34.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.396+0000 7f57ea20b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f57e4142430 0x7f57e41428c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:34.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.396+0000 7f57ea20b640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f57e406fa50 con 0x7f57e4142430 2024-03-18T15:34:34.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.396+0000 7f57ea20b640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f57e4070010 con 0x7f57e40744d0 2024-03-18T15:34:34.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.396+0000 7f57ea20b640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f57e40b55f0 con 0x7f57e40b3050 2024-03-18T15:34:34.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.396+0000 7f57e3fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f57e4142430 0x7f57e41428c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:34.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.396+0000 7f57e3fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f57e4142430 0x7f57e41428c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56496/0 (socket says 172.21.15.5:56496) 2024-03-18T15:34:34.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.396+0000 7f57e3fff640 1 -- 172.21.15.5:0/3843689742 learned_addr learned my addr 172.21.15.5:0/3843689742 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:34.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.396+0000 7f57e2ffd640 1 --2- 172.21.15.5:0/3843689742 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f57e40b3050 0x7f57e4145af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:34.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.396+0000 7f57e37fe640 1 --2- 172.21.15.5:0/3843689742 >> v2:172.21.15.80:3300/0 conn(0x7f57e40744d0 0x7f57e41455b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:34.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.396+0000 7f57e3fff640 1 -- 172.21.15.5:0/3843689742 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f57e40b3050 msgr2=0x7f57e4145af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:34.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.396+0000 7f57e3fff640 1 --2- 172.21.15.5:0/3843689742 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f57e40b3050 0x7f57e4145af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:34.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.396+0000 7f57e3fff640 1 -- 172.21.15.5:0/3843689742 >> v2:172.21.15.80:3300/0 conn(0x7f57e40744d0 msgr2=0x7f57e41455b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:34.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.396+0000 7f57e3fff640 1 --2- 172.21.15.5:0/3843689742 >> v2:172.21.15.80:3300/0 conn(0x7f57e40744d0 0x7f57e41455b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:34.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.396+0000 7f57e3fff640 1 -- 172.21.15.5:0/3843689742 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57e4142e00 con 0x7f57e4142430 2024-03-18T15:34:34.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.396+0000 7f57e37fe640 1 --2- 172.21.15.5:0/3843689742 >> v2:172.21.15.80:3300/0 conn(0x7f57e40744d0 0x7f57e41455b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:34:34.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.400+0000 7f57e3fff640 1 --2- 172.21.15.5:0/3843689742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57e4142430 0x7f57e41428c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f57e40b9ad0 tx=0x7f57dc01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3e16f3c3b19973a server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:34.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.400+0000 7f57e27fc640 1 -- 172.21.15.5:0/3843689742 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57dc0a56f0 con 0x7f57e4142430 2024-03-18T15:34:34.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.400+0000 7f57e27fc640 1 -- 172.21.15.5:0/3843689742 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57dc0a5890 con 0x7f57e4142430 2024-03-18T15:34:34.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.400+0000 7f57e27fc640 1 -- 172.21.15.5:0/3843689742 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57dc0a5c60 con 0x7f57e4142430 2024-03-18T15:34:34.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.400+0000 7f57ea20b640 1 -- 172.21.15.5:0/3843689742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57e4142430 msgr2=0x7f57e41428c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:34.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.400+0000 7f57ea20b640 1 --2- 172.21.15.5:0/3843689742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57e4142430 0x7f57e41428c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f57e40b9ad0 tx=0x7f57dc01c960 comp rx=0 tx=0).stop 2024-03-18T15:34:34.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.400+0000 7f57ea20b640 1 -- 172.21.15.5:0/3843689742 shutdown_connections 2024-03-18T15:34:34.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.400+0000 7f57ea20b640 1 --2- 172.21.15.5:0/3843689742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57e4142430 0x7f57e41428c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:34.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.400+0000 7f57ea20b640 1 --2- 172.21.15.5:0/3843689742 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f57e40b3050 0x7f57e4145af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:34.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.400+0000 7f57ea20b640 1 --2- 172.21.15.5:0/3843689742 >> v2:172.21.15.80:3300/0 conn(0x7f57e40744d0 0x7f57e41455b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:34.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.400+0000 7f57ea20b640 1 -- 172.21.15.5:0/3843689742 >> 172.21.15.5:0/3843689742 conn(0x7f57e406bad0 msgr2=0x7f57e40b9190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:34.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.400+0000 7f57ea20b640 1 -- 172.21.15.5:0/3843689742 shutdown_connections 2024-03-18T15:34:34.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.400+0000 7f57ea20b640 1 -- 172.21.15.5:0/3843689742 wait complete. 2024-03-18T15:34:34.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.400+0000 7f57ea20b640 1 Processor -- start 2024-03-18T15:34:34.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.404+0000 7f57ea20b640 1 -- start start 2024-03-18T15:34:34.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.404+0000 7f57ea20b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57e40744d0 0x7f57e413b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:34.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.404+0000 7f57ea20b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57e40b3050 0x7f57e41e07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:34.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.404+0000 7f57ea20b640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f57e4142430 0x7f57e40b0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:34.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.404+0000 7f57ea20b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f57e4149110 con 0x7f57e40744d0 2024-03-18T15:34:34.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.404+0000 7f57ea20b640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f57e4070f10 con 0x7f57e40b3050 2024-03-18T15:34:34.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.404+0000 7f57ea20b640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f57e4149290 con 0x7f57e4142430 2024-03-18T15:34:34.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.404+0000 7f57e37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57e40744d0 0x7f57e413b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:34.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.404+0000 7f57e37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57e40744d0 0x7f57e413b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56500/0 (socket says 172.21.15.5:56500) 2024-03-18T15:34:34.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.404+0000 7f57e37fe640 1 -- 172.21.15.5:0/3409289872 learned_addr learned my addr 172.21.15.5:0/3409289872 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:34.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.404+0000 7f57e2ffd640 1 --2- 172.21.15.5:0/3409289872 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57e40b3050 0x7f57e41e07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:34.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.404+0000 7f57e3fff640 1 --2- 172.21.15.5:0/3409289872 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f57e4142430 0x7f57e40b0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:34.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.404+0000 7f57e37fe640 1 -- 172.21.15.5:0/3409289872 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f57e4142430 msgr2=0x7f57e40b0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:34.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.404+0000 7f57e37fe640 1 --2- 172.21.15.5:0/3409289872 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f57e4142430 0x7f57e40b0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:34.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.404+0000 7f57e37fe640 1 -- 172.21.15.5:0/3409289872 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57e40b3050 msgr2=0x7f57e41e07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:34.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.404+0000 7f57e37fe640 1 --2- 172.21.15.5:0/3409289872 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57e40b3050 0x7f57e41e07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:34.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.404+0000 7f57e37fe640 1 -- 172.21.15.5:0/3409289872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57e41dfcf0 con 0x7f57e40744d0 2024-03-18T15:34:34.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.404+0000 7f57e3fff640 1 --2- 172.21.15.5:0/3409289872 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f57e4142430 0x7f57e40b0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:34.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.404+0000 7f57e37fe640 1 --2- 172.21.15.5:0/3409289872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57e40744d0 0x7f57e413b400 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f57d400e810 tx=0x7f57d400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:34.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.404+0000 7f57cbfff640 1 -- 172.21.15.5:0/3409289872 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d400f040 con 0x7f57e40744d0 2024-03-18T15:34:34.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.404+0000 7f57cbfff640 1 -- 172.21.15.5:0/3409289872 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57d4004030 con 0x7f57e40744d0 2024-03-18T15:34:34.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.404+0000 7f57ea20b640 1 -- 172.21.15.5:0/3409289872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57e40b1ee0 con 0x7f57e40744d0 2024-03-18T15:34:34.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.404+0000 7f57cbfff640 1 -- 172.21.15.5:0/3409289872 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d4004400 con 0x7f57e40744d0 2024-03-18T15:34:34.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.404+0000 7f57ea20b640 1 -- 172.21.15.5:0/3409289872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57e41dffb0 con 0x7f57e40744d0 2024-03-18T15:34:34.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.404+0000 7f57ea20b640 1 -- 172.21.15.5:0/3409289872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57e404f550 con 0x7f57e40744d0 2024-03-18T15:34:34.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.404+0000 7f57cbfff640 1 -- 172.21.15.5:0/3409289872 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f57d400c6d0 con 0x7f57e40744d0 2024-03-18T15:34:34.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.404+0000 7f57cbfff640 1 --2- 172.21.15.5:0/3409289872 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f57d0067660 0x7f57d0069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:34.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.404+0000 7f57cbfff640 1 -- 172.21.15.5:0/3409289872 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7f57d4085530 con 0x7f57e40744d0 2024-03-18T15:34:34.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.404+0000 7f57e2ffd640 1 --2- 172.21.15.5:0/3409289872 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f57d0067660 0x7f57d0069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:34.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.408+0000 7f57e2ffd640 1 --2- 172.21.15.5:0/3409289872 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f57d0067660 0x7f57d0069b20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f57e4139fb0 tx=0x7f57d8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:34.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.408+0000 7f57cbfff640 1 -- 172.21.15.5:0/3409289872 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f57d4017020 con 0x7f57e40744d0 2024-03-18T15:34:34.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.572+0000 7f57ea20b640 1 -- 172.21.15.5:0/3409289872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f57e4070950 con 0x7f57e40744d0 2024-03-18T15:34:34.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.572+0000 7f57cbfff640 1 -- 172.21.15.5:0/3409289872 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) ==== 93+0+10 (secure 0 0 0) 0x7f57d40561a0 con 0x7f57e40744d0 2024-03-18T15:34:34.575 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-03-18T15:34:34.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.572+0000 7f57ea20b640 1 -- 172.21.15.5:0/3409289872 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f57d0067660 msgr2=0x7f57d0069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:34.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.572+0000 7f57ea20b640 1 --2- 172.21.15.5:0/3409289872 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f57d0067660 0x7f57d0069b20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f57e4139fb0 tx=0x7f57d8008040 comp rx=0 tx=0).stop 2024-03-18T15:34:34.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.572+0000 7f57ea20b640 1 -- 172.21.15.5:0/3409289872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57e40744d0 msgr2=0x7f57e413b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:34.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.572+0000 7f57ea20b640 1 --2- 172.21.15.5:0/3409289872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57e40744d0 0x7f57e413b400 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f57d400e810 tx=0x7f57d400ecd0 comp rx=0 tx=0).stop 2024-03-18T15:34:34.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.576+0000 7f57ea20b640 1 -- 172.21.15.5:0/3409289872 shutdown_connections 2024-03-18T15:34:34.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.576+0000 7f57ea20b640 1 --2- 172.21.15.5:0/3409289872 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f57d0067660 0x7f57d0069b20 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:34.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.576+0000 7f57ea20b640 1 --2- 172.21.15.5:0/3409289872 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f57e4142430 0x7f57e40b0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:34.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.576+0000 7f57ea20b640 1 --2- 172.21.15.5:0/3409289872 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57e40b3050 0x7f57e41e07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:34.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.576+0000 7f57ea20b640 1 --2- 172.21.15.5:0/3409289872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57e40744d0 0x7f57e413b400 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:34.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.576+0000 7f57ea20b640 1 -- 172.21.15.5:0/3409289872 >> 172.21.15.5:0/3409289872 conn(0x7f57e406bad0 msgr2=0x7f57e40b3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:34.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.576+0000 7f57ea20b640 1 -- 172.21.15.5:0/3409289872 shutdown_connections 2024-03-18T15:34:34.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.576+0000 7f57ea20b640 1 -- 172.21.15.5:0/3409289872 wait complete. 2024-03-18T15:34:34.594 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-18T15:34:34.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.712+0000 7fded62f8640 1 Processor -- start 2024-03-18T15:34:34.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.712+0000 7fded62f8640 1 -- start start 2024-03-18T15:34:34.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.712+0000 7fded62f8640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fded01418d0 0x7fded014ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:34.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.712+0000 7fded62f8640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fded00ad8e0 0x7fded014b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:34.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.712+0000 7fded62f8640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fded0074930 con 0x7fded0074550 2024-03-18T15:34:34.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.712+0000 7fded62f8640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fded00ae920 con 0x7fded00ad8e0 2024-03-18T15:34:34.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.712+0000 7fded62f8640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fded00aeb70 con 0x7fded01418d0 2024-03-18T15:34:34.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.712+0000 7fded4895640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fded00ad8e0 0x7fded014b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:34.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.712+0000 7fded4895640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fded00ad8e0 0x7fded014b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56516/0 (socket says 172.21.15.5:56516) 2024-03-18T15:34:34.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.712+0000 7fded4895640 1 -- 172.21.15.5:0/1512716605 learned_addr learned my addr 172.21.15.5:0/1512716605 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:34.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.712+0000 7fdecf7fe640 1 --2- 172.21.15.5:0/1512716605 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fded01418d0 0x7fded014ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:34.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.712+0000 7fded4895640 1 -- 172.21.15.5:0/1512716605 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fded01418d0 msgr2=0x7fded014ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:34.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.712+0000 7fded4895640 1 --2- 172.21.15.5:0/1512716605 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fded01418d0 0x7fded014ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:34.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.712+0000 7fded4895640 1 -- 172.21.15.5:0/1512716605 >> v1:172.21.15.80:6789/0 conn(0x7fded0074550 legacy=0x7fded01411c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:34.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.712+0000 7fded4895640 1 -- 172.21.15.5:0/1512716605 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fded00aee50 con 0x7fded00ad8e0 2024-03-18T15:34:34.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.712+0000 7fdecf7fe640 1 --2- 172.21.15.5:0/1512716605 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fded01418d0 0x7fded014ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:34:34.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.716+0000 7fded4895640 1 --2- 172.21.15.5:0/1512716605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded00ad8e0 0x7fded014b2a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fdec8009e50 tx=0x7fdec801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8461d11ed24b61a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:34.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.716+0000 7fdeceffd640 1 -- 172.21.15.5:0/1512716605 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec80a5770 con 0x7fded00ad8e0 2024-03-18T15:34:34.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.716+0000 7fdeceffd640 1 -- 172.21.15.5:0/1512716605 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdec80a5910 con 0x7fded00ad8e0 2024-03-18T15:34:34.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.716+0000 7fdeceffd640 1 -- 172.21.15.5:0/1512716605 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec80a5ce0 con 0x7fded00ad8e0 2024-03-18T15:34:34.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.716+0000 7fded62f8640 1 -- 172.21.15.5:0/1512716605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded00ad8e0 msgr2=0x7fded014b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:34.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.716+0000 7fded62f8640 1 --2- 172.21.15.5:0/1512716605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded00ad8e0 0x7fded014b2a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fdec8009e50 tx=0x7fdec801c9a0 comp rx=0 tx=0).stop 2024-03-18T15:34:34.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.716+0000 7fded62f8640 1 -- 172.21.15.5:0/1512716605 shutdown_connections 2024-03-18T15:34:34.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.716+0000 7fded62f8640 1 --2- 172.21.15.5:0/1512716605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded00ad8e0 0x7fded014b2a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:34.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.716+0000 7fded62f8640 1 --2- 172.21.15.5:0/1512716605 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fded01418d0 0x7fded014ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:34.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.716+0000 7fded62f8640 1 -- 172.21.15.5:0/1512716605 >> 172.21.15.5:0/1512716605 conn(0x7fded00b5d00 msgr2=0x7fded00b6110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:34.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.716+0000 7fded62f8640 1 -- 172.21.15.5:0/1512716605 shutdown_connections 2024-03-18T15:34:34.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.716+0000 7fded62f8640 1 -- 172.21.15.5:0/1512716605 wait complete. 2024-03-18T15:34:34.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.716+0000 7fded62f8640 1 Processor -- start 2024-03-18T15:34:34.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.716+0000 7fded62f8640 1 -- start start 2024-03-18T15:34:34.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.716+0000 7fded62f8640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fded0074550 0x7fded01e2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:34.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.716+0000 7fded62f8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded00ad8e0 0x7fded00b77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:34.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.716+0000 7fded62f8640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fded01418d0 0x7fded00b98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:34.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.716+0000 7fded62f8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fded01548c0 con 0x7fded00ad8e0 2024-03-18T15:34:34.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.716+0000 7fded62f8640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fded0154740 con 0x7fded01418d0 2024-03-18T15:34:34.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.716+0000 7fded62f8640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fded0154a40 con 0x7fded0074550 2024-03-18T15:34:34.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.716+0000 7fdecf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded00ad8e0 0x7fded00b77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:34.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.716+0000 7fdecf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded00ad8e0 0x7fded00b77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56524/0 (socket says 172.21.15.5:56524) 2024-03-18T15:34:34.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.716+0000 7fdecf7fe640 1 -- 172.21.15.5:0/380469421 learned_addr learned my addr 172.21.15.5:0/380469421 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:34.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.716+0000 7fdecffff640 1 --2- 172.21.15.5:0/380469421 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fded0074550 0x7fded01e2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:34.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.716+0000 7fded4895640 1 --2- 172.21.15.5:0/380469421 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fded01418d0 0x7fded00b98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:34.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.716+0000 7fdecf7fe640 1 -- 172.21.15.5:0/380469421 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fded0074550 msgr2=0x7fded01e2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:34.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.716+0000 7fdecf7fe640 1 --2- 172.21.15.5:0/380469421 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fded0074550 0x7fded01e2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:34.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.716+0000 7fdecf7fe640 1 -- 172.21.15.5:0/380469421 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fded01418d0 msgr2=0x7fded00b98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:34.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.716+0000 7fdecf7fe640 1 --2- 172.21.15.5:0/380469421 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fded01418d0 0x7fded00b98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:34.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.716+0000 7fdecf7fe640 1 -- 172.21.15.5:0/380469421 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fded0148020 con 0x7fded00ad8e0 2024-03-18T15:34:34.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.716+0000 7fded4895640 1 --2- 172.21.15.5:0/380469421 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fded01418d0 0x7fded00b98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:34.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.716+0000 7fdecf7fe640 1 --2- 172.21.15.5:0/380469421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded00ad8e0 0x7fded00b77c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fdec400d800 tx=0x7fdec400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:34.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.716+0000 7fdeccff9640 1 -- 172.21.15.5:0/380469421 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec400b6c0 con 0x7fded00ad8e0 2024-03-18T15:34:34.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.716+0000 7fdeccff9640 1 -- 172.21.15.5:0/380469421 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdec4019070 con 0x7fded00ad8e0 2024-03-18T15:34:34.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.716+0000 7fded62f8640 1 -- 172.21.15.5:0/380469421 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fded0141f90 con 0x7fded00ad8e0 2024-03-18T15:34:34.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.716+0000 7fdeccff9640 1 -- 172.21.15.5:0/380469421 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec400ba80 con 0x7fded00ad8e0 2024-03-18T15:34:34.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.716+0000 7fded62f8640 1 -- 172.21.15.5:0/380469421 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fded00b7d00 con 0x7fded00ad8e0 2024-03-18T15:34:34.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.720+0000 7fded62f8640 1 -- 172.21.15.5:0/380469421 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fded004f550 con 0x7fded00ad8e0 2024-03-18T15:34:34.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.720+0000 7fdeccff9640 1 -- 172.21.15.5:0/380469421 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fdec400bc20 con 0x7fded00ad8e0 2024-03-18T15:34:34.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.720+0000 7fdeccff9640 1 --2- 172.21.15.5:0/380469421 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fdebc067660 0x7fdebc069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:34.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.720+0000 7fdeccff9640 1 -- 172.21.15.5:0/380469421 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7fdec4085c10 con 0x7fded00ad8e0 2024-03-18T15:34:34.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.720+0000 7fdecffff640 1 --2- 172.21.15.5:0/380469421 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fdebc067660 0x7fdebc069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:34.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.720+0000 7fdecffff640 1 --2- 172.21.15.5:0/380469421 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fdebc067660 0x7fdebc069b20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdec0001650 tx=0x7fdec0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:34.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.724+0000 7fdeccff9640 1 -- 172.21.15.5:0/380469421 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fdec4056920 con 0x7fded00ad8e0 2024-03-18T15:34:34.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.884+0000 7fded62f8640 1 -- 172.21.15.5:0/380469421 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fded006c290 con 0x7fded00ad8e0 2024-03-18T15:34:34.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.884+0000 7fdeccff9640 1 -- 172.21.15.5:0/380469421 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) ==== 104+0+11 (secure 0 0 0) 0x7fdec4056220 con 0x7fded00ad8e0 2024-03-18T15:34:34.888 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-03-18T15:34:34.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.888+0000 7fded62f8640 1 -- 172.21.15.5:0/380469421 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fdebc067660 msgr2=0x7fdebc069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:34.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.888+0000 7fded62f8640 1 --2- 172.21.15.5:0/380469421 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fdebc067660 0x7fdebc069b20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdec0001650 tx=0x7fdec0009040 comp rx=0 tx=0).stop 2024-03-18T15:34:34.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.888+0000 7fded62f8640 1 -- 172.21.15.5:0/380469421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded00ad8e0 msgr2=0x7fded00b77c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:34.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.888+0000 7fded62f8640 1 --2- 172.21.15.5:0/380469421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded00ad8e0 0x7fded00b77c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fdec400d800 tx=0x7fdec400dcc0 comp rx=0 tx=0).stop 2024-03-18T15:34:34.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.888+0000 7fded62f8640 1 -- 172.21.15.5:0/380469421 shutdown_connections 2024-03-18T15:34:34.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.888+0000 7fded62f8640 1 --2- 172.21.15.5:0/380469421 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fdebc067660 0x7fdebc069b20 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:34.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.888+0000 7fded62f8640 1 --2- 172.21.15.5:0/380469421 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fded01418d0 0x7fded00b98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:34.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.888+0000 7fded62f8640 1 --2- 172.21.15.5:0/380469421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded00ad8e0 0x7fded00b77c0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:34.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.888+0000 7fded62f8640 1 --2- 172.21.15.5:0/380469421 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fded0074550 0x7fded01e2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:34.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.888+0000 7fded62f8640 1 -- 172.21.15.5:0/380469421 >> 172.21.15.5:0/380469421 conn(0x7fded00b5d00 msgr2=0x7fded013c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:34.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.888+0000 7fded62f8640 1 -- 172.21.15.5:0/380469421 shutdown_connections 2024-03-18T15:34:34.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:34.888+0000 7fded62f8640 1 -- 172.21.15.5:0/380469421 wait complete. 2024-03-18T15:34:34.908 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-18T15:34:35.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.032+0000 7fc020e55640 1 Processor -- start 2024-03-18T15:34:35.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.032+0000 7fc020e55640 1 -- start start 2024-03-18T15:34:35.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.032+0000 7fc020e55640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc01c1479a0 0x7fc01c147d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:35.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.032+0000 7fc020e55640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc01c1482c0 0x7fc01c070d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:35.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.032+0000 7fc020e55640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc01c0716d0 con 0x7fc01c071240 2024-03-18T15:34:35.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.032+0000 7fc020e55640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fc01c0702b0 con 0x7fc01c1482c0 2024-03-18T15:34:35.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.032+0000 7fc020e55640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fc01c070870 con 0x7fc01c1479a0 2024-03-18T15:34:35.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.032+0000 7fc01ad9d640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fc01c071240 0x7fc01c13dfa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44308/0 (socket says 172.21.15.5:44308) 2024-03-18T15:34:35.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.032+0000 7fc01ad9d640 1 -- 172.21.15.5:0/3840416875 learned_addr learned my addr 172.21.15.5:0/3840416875 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:35.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.032+0000 7fc019d9b640 1 --2- 172.21.15.5:0/3840416875 >> v2:172.21.15.80:3300/0 conn(0x7fc01c1482c0 0x7fc01c070d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:35.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.032+0000 7fc01a59c640 1 --2- 172.21.15.5:0/3840416875 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc01c1479a0 0x7fc01c147d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:35.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.032+0000 7fc01959a640 1 -- 172.21.15.5:0/3840416875 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 186774945 0 0) 0x7fc01c0716d0 con 0x7fc01c071240 2024-03-18T15:34:35.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.032+0000 7fc01959a640 1 -- 172.21.15.5:0/3840416875 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc004003530 con 0x7fc01c071240 2024-03-18T15:34:35.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.032+0000 7fc01959a640 1 -- 172.21.15.5:0/3840416875 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1309589815 0 0) 0x7fc004003530 con 0x7fc01c071240 2024-03-18T15:34:35.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.032+0000 7fc01959a640 1 -- 172.21.15.5:0/3840416875 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc01c1479a0 msgr2=0x7fc01c147d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:35.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.032+0000 7fc01959a640 1 --2- 172.21.15.5:0/3840416875 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc01c1479a0 0x7fc01c147d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:35.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.032+0000 7fc01959a640 1 -- 172.21.15.5:0/3840416875 >> v2:172.21.15.80:3300/0 conn(0x7fc01c1482c0 msgr2=0x7fc01c070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:35.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.032+0000 7fc01959a640 1 --2- 172.21.15.5:0/3840416875 >> v2:172.21.15.80:3300/0 conn(0x7fc01c1482c0 0x7fc01c070d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:35.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.032+0000 7fc01959a640 1 -- 172.21.15.5:0/3840416875 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc01c072a70 con 0x7fc01c071240 2024-03-18T15:34:35.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.032+0000 7fc01959a640 1 -- 172.21.15.5:0/3840416875 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3837549003 0 0) 0x7fc014001a00 con 0x7fc01c071240 2024-03-18T15:34:35.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.032+0000 7fc01959a640 1 -- 172.21.15.5:0/3840416875 >> v1:172.21.15.5:6789/0 conn(0x7fc01c071240 legacy=0x7fc01c13dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:35.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.032+0000 7fc01959a640 1 --2- 172.21.15.5:0/3840416875 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc004003aa0 0x7fc004003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:35.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.032+0000 7fc01959a640 1 --2- 172.21.15.5:0/3840416875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc004004810 0x7fc004004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:35.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.032+0000 7fc01959a640 1 --2- 172.21.15.5:0/3840416875 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc01c1479a0 0x7fc0040051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:35.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.032+0000 7fc01959a640 1 -- 172.21.15.5:0/3840416875 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c0702b0 con 0x7fc004004810 2024-03-18T15:34:35.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.032+0000 7fc01959a640 1 -- 172.21.15.5:0/3840416875 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c070870 con 0x7fc004003aa0 2024-03-18T15:34:35.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.032+0000 7fc01959a640 1 -- 172.21.15.5:0/3840416875 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fc00400bb50 con 0x7fc01c1479a0 2024-03-18T15:34:35.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.032+0000 7fc019d9b640 1 --2- 172.21.15.5:0/3840416875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc004004810 0x7fc004004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:35.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.032+0000 7fc01a59c640 1 --2- 172.21.15.5:0/3840416875 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc004003aa0 0x7fc004003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:35.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.032+0000 7fc01ad9d640 1 --2- 172.21.15.5:0/3840416875 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc01c1479a0 0x7fc0040051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:35.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.032+0000 7fc019d9b640 1 -- 172.21.15.5:0/3840416875 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc01c1479a0 msgr2=0x7fc0040051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:35.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.032+0000 7fc019d9b640 1 --2- 172.21.15.5:0/3840416875 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc01c1479a0 0x7fc0040051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:35.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.032+0000 7fc019d9b640 1 -- 172.21.15.5:0/3840416875 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc004003aa0 msgr2=0x7fc004003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:35.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.032+0000 7fc019d9b640 1 --2- 172.21.15.5:0/3840416875 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc004003aa0 0x7fc004003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:35.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.032+0000 7fc019d9b640 1 -- 172.21.15.5:0/3840416875 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc004003530 con 0x7fc004004810 2024-03-18T15:34:35.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.032+0000 7fc01ad9d640 1 --2- 172.21.15.5:0/3840416875 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc01c1479a0 0x7fc0040051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:35.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.036+0000 7fc019d9b640 1 --2- 172.21.15.5:0/3840416875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc004004810 0x7fc004004c90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc010009ae0 tx=0x7fc01001cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df5e0bbd3134d2e3 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:35.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.036+0000 7fc01959a640 1 -- 172.21.15.5:0/3840416875 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0100096b0 con 0x7fc004004810 2024-03-18T15:34:35.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.036+0000 7fc01959a640 1 -- 172.21.15.5:0/3840416875 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0100a64c0 con 0x7fc004004810 2024-03-18T15:34:35.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.036+0000 7fc01959a640 1 -- 172.21.15.5:0/3840416875 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0100a68b0 con 0x7fc004004810 2024-03-18T15:34:35.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.036+0000 7fc020e55640 1 -- 172.21.15.5:0/3840416875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc004004810 msgr2=0x7fc004004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:35.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.036+0000 7fc020e55640 1 --2- 172.21.15.5:0/3840416875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc004004810 0x7fc004004c90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc010009ae0 tx=0x7fc01001cab0 comp rx=0 tx=0).stop 2024-03-18T15:34:35.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.036+0000 7fc01ad9d640 1 -- 172.21.15.5:0/3840416875 reap_dead start 2024-03-18T15:34:35.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.036+0000 7fc020e55640 1 -- 172.21.15.5:0/3840416875 shutdown_connections 2024-03-18T15:34:35.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.036+0000 7fc020e55640 1 -- 172.21.15.5:0/3840416875 >> 172.21.15.5:0/3840416875 conn(0x7fc01c0b7a80 msgr2=0x7fc01c0b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:35.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.036+0000 7fc020e55640 1 -- 172.21.15.5:0/3840416875 shutdown_connections 2024-03-18T15:34:35.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.036+0000 7fc020e55640 1 -- 172.21.15.5:0/3840416875 wait complete. 2024-03-18T15:34:35.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.036+0000 7fc020e55640 1 Processor -- start 2024-03-18T15:34:35.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.036+0000 7fc020e55640 1 -- start start 2024-03-18T15:34:35.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.036+0000 7fc020e55640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc004004810 0x7fc01c14ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:35.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.036+0000 7fc020e55640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc01c14cbb0 0x7fc01c1de3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:35.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.036+0000 7fc020e55640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc01c1de900 0x7fc01c13adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:35.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.036+0000 7fc020e55640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c14f610 con 0x7fc01c14cbb0 2024-03-18T15:34:35.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.036+0000 7fc020e55640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c14f490 con 0x7fc004004810 2024-03-18T15:34:35.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.036+0000 7fc020e55640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fc01c14f790 con 0x7fc01c1de900 2024-03-18T15:34:35.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.040+0000 7fc019d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc01c14cbb0 0x7fc01c1de3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:35.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.040+0000 7fc019d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc01c14cbb0 0x7fc01c1de3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56538/0 (socket says 172.21.15.5:56538) 2024-03-18T15:34:35.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.040+0000 7fc019d9b640 1 -- 172.21.15.5:0/2475565617 learned_addr learned my addr 172.21.15.5:0/2475565617 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:35.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.040+0000 7fc01ad9d640 1 --2- 172.21.15.5:0/2475565617 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc01c1de900 0x7fc01c13adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:35.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.040+0000 7fc01a59c640 1 --2- 172.21.15.5:0/2475565617 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc004004810 0x7fc01c14ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:35.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.040+0000 7fc019d9b640 1 -- 172.21.15.5:0/2475565617 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc01c1de900 msgr2=0x7fc01c13adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:35.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.040+0000 7fc019d9b640 1 --2- 172.21.15.5:0/2475565617 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc01c1de900 0x7fc01c13adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:35.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.040+0000 7fc019d9b640 1 -- 172.21.15.5:0/2475565617 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc004004810 msgr2=0x7fc01c14ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:35.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.040+0000 7fc019d9b640 1 --2- 172.21.15.5:0/2475565617 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc004004810 0x7fc01c14ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:35.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.040+0000 7fc019d9b640 1 -- 172.21.15.5:0/2475565617 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc01c0b1620 con 0x7fc01c14cbb0 2024-03-18T15:34:35.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.040+0000 7fc01a59c640 1 --2- 172.21.15.5:0/2475565617 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc004004810 0x7fc01c14ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:34:35.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.040+0000 7fc019d9b640 1 --2- 172.21.15.5:0/2475565617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc01c14cbb0 0x7fc01c1de3c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc01001c8e0 tx=0x7fc004005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:35.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.040+0000 7fc00affd640 1 -- 172.21.15.5:0/2475565617 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0100bd020 con 0x7fc01c14cbb0 2024-03-18T15:34:35.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.040+0000 7fc00affd640 1 -- 172.21.15.5:0/2475565617 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0100a6670 con 0x7fc01c14cbb0 2024-03-18T15:34:35.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.040+0000 7fc00affd640 1 -- 172.21.15.5:0/2475565617 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0100a6a40 con 0x7fc01c14cbb0 2024-03-18T15:34:35.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.040+0000 7fc020e55640 1 -- 172.21.15.5:0/2475565617 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc01c14d1a0 con 0x7fc01c14cbb0 2024-03-18T15:34:35.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.040+0000 7fc020e55640 1 -- 172.21.15.5:0/2475565617 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc01c0b1890 con 0x7fc01c14cbb0 2024-03-18T15:34:35.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.040+0000 7fc020e55640 1 -- 172.21.15.5:0/2475565617 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc01c1ded10 con 0x7fc01c14cbb0 2024-03-18T15:34:35.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.040+0000 7fc00affd640 1 -- 172.21.15.5:0/2475565617 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fc0100c0020 con 0x7fc01c14cbb0 2024-03-18T15:34:35.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.040+0000 7fc00affd640 1 --2- 172.21.15.5:0/2475565617 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fbffc067690 0x7fbffc069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:35.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.040+0000 7fc00affd640 1 -- 172.21.15.5:0/2475565617 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7fc0101204f0 con 0x7fc01c14cbb0 2024-03-18T15:34:35.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.040+0000 7fc01a59c640 1 --2- 172.21.15.5:0/2475565617 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fbffc067690 0x7fbffc069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:35.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.040+0000 7fc01a59c640 1 --2- 172.21.15.5:0/2475565617 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fbffc067690 0x7fbffc069b50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc00c009a10 tx=0x7fc00c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:35.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.044+0000 7fc00affd640 1 -- 172.21.15.5:0/2475565617 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fc01c1ded10 con 0x7fc01c14cbb0 2024-03-18T15:34:35.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.204+0000 7fc020e55640 1 -- 172.21.15.5:0/2475565617 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fc01c04f550 con 0x7fc01c14cbb0 2024-03-18T15:34:35.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.204+0000 7fc00affd640 1 -- 172.21.15.5:0/2475565617 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) ==== 100+0+11 (secure 0 0 0) 0x7fc0100c7030 con 0x7fc01c14cbb0 2024-03-18T15:34:35.208 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-03-18T15:34:35.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.208+0000 7fc020e55640 1 -- 172.21.15.5:0/2475565617 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fbffc067690 msgr2=0x7fbffc069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:35.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.208+0000 7fc020e55640 1 --2- 172.21.15.5:0/2475565617 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fbffc067690 0x7fbffc069b50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc00c009a10 tx=0x7fc00c008040 comp rx=0 tx=0).stop 2024-03-18T15:34:35.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.208+0000 7fc020e55640 1 -- 172.21.15.5:0/2475565617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc01c14cbb0 msgr2=0x7fc01c1de3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:35.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.208+0000 7fc020e55640 1 --2- 172.21.15.5:0/2475565617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc01c14cbb0 0x7fc01c1de3c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc01001c8e0 tx=0x7fc004005840 comp rx=0 tx=0).stop 2024-03-18T15:34:35.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.208+0000 7fc020e55640 1 -- 172.21.15.5:0/2475565617 shutdown_connections 2024-03-18T15:34:35.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.208+0000 7fc020e55640 1 --2- 172.21.15.5:0/2475565617 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fbffc067690 0x7fbffc069b50 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:35.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.208+0000 7fc020e55640 1 --2- 172.21.15.5:0/2475565617 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fc01c1de900 0x7fc01c13adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:35.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.208+0000 7fc020e55640 1 --2- 172.21.15.5:0/2475565617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc01c14cbb0 0x7fc01c1de3c0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:35.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.208+0000 7fc020e55640 1 --2- 172.21.15.5:0/2475565617 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc004004810 0x7fc01c14ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:35.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.208+0000 7fc020e55640 1 -- 172.21.15.5:0/2475565617 >> 172.21.15.5:0/2475565617 conn(0x7fc01c0b7a80 msgr2=0x7fc01c144350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:35.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.208+0000 7fc020e55640 1 -- 172.21.15.5:0/2475565617 shutdown_connections 2024-03-18T15:34:35.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.208+0000 7fc020e55640 1 -- 172.21.15.5:0/2475565617 wait complete. 2024-03-18T15:34:35.228 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:34:35.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.352+0000 7f08b6ad8640 1 Processor -- start 2024-03-18T15:34:35.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.352+0000 7f08b6ad8640 1 -- start start 2024-03-18T15:34:35.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.352+0000 7f08b6ad8640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f08b00b6c60 0x7f08b00b45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:35.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.352+0000 7f08b6ad8640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f08b0072b20 0x7f08b0148870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:35.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.352+0000 7f08b6ad8640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08b00b6020 con 0x7f08b00b6520 2024-03-18T15:34:35.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.352+0000 7f08b6ad8640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f08b00b79b0 con 0x7f08b0072b20 2024-03-18T15:34:35.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.352+0000 7f08b6ad8640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f08b00b76a0 con 0x7f08b00b6c60 2024-03-18T15:34:35.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.352+0000 7f08affff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f08b0072b20 0x7f08b0148870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:35.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.352+0000 7f08affff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f08b0072b20 0x7f08b0148870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56546/0 (socket says 172.21.15.5:56546) 2024-03-18T15:34:35.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.352+0000 7f08affff640 1 -- 172.21.15.5:0/3770842044 learned_addr learned my addr 172.21.15.5:0/3770842044 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:35.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.352+0000 7f08b4874640 1 --2- 172.21.15.5:0/3770842044 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f08b00b6c60 0x7f08b00b45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:35.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.352+0000 7f08affff640 1 -- 172.21.15.5:0/3770842044 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f08b00b6c60 msgr2=0x7f08b00b45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:35.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.352+0000 7f08affff640 1 --2- 172.21.15.5:0/3770842044 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f08b00b6c60 0x7f08b00b45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:35.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.352+0000 7f08affff640 1 -- 172.21.15.5:0/3770842044 >> v1:172.21.15.80:6789/0 conn(0x7f08b00b6520 legacy=0x7f08b0071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:35.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.352+0000 7f08affff640 1 -- 172.21.15.5:0/3770842044 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08b00b7040 con 0x7f08b0072b20 2024-03-18T15:34:35.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.356+0000 7f08affff640 1 --2- 172.21.15.5:0/3770842044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08b0072b20 0x7f08b0148870 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f08a4009870 tx=0x7f08a401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cea923fe4a77300 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:35.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.356+0000 7f08af7fe640 1 -- 172.21.15.5:0/3770842044 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a40aa650 con 0x7f08b0072b20 2024-03-18T15:34:35.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.356+0000 7f08af7fe640 1 -- 172.21.15.5:0/3770842044 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08a40af070 con 0x7f08b0072b20 2024-03-18T15:34:35.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.356+0000 7f08af7fe640 1 -- 172.21.15.5:0/3770842044 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a40aaa10 con 0x7f08b0072b20 2024-03-18T15:34:35.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.356+0000 7f08b6ad8640 1 -- 172.21.15.5:0/3770842044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08b0072b20 msgr2=0x7f08b0148870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:35.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.356+0000 7f08b6ad8640 1 --2- 172.21.15.5:0/3770842044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08b0072b20 0x7f08b0148870 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f08a4009870 tx=0x7f08a401c4d0 comp rx=0 tx=0).stop 2024-03-18T15:34:35.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.356+0000 7f08b6ad8640 1 -- 172.21.15.5:0/3770842044 shutdown_connections 2024-03-18T15:34:35.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.356+0000 7f08b6ad8640 1 --2- 172.21.15.5:0/3770842044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08b0072b20 0x7f08b0148870 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:35.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.356+0000 7f08b6ad8640 1 --2- 172.21.15.5:0/3770842044 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f08b00b6c60 0x7f08b00b45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:35.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.356+0000 7f08b6ad8640 1 -- 172.21.15.5:0/3770842044 >> 172.21.15.5:0/3770842044 conn(0x7f08b0066fc0 msgr2=0x7f08b00673d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:35.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.356+0000 7f08b6ad8640 1 -- 172.21.15.5:0/3770842044 shutdown_connections 2024-03-18T15:34:35.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.356+0000 7f08b6ad8640 1 -- 172.21.15.5:0/3770842044 wait complete. 2024-03-18T15:34:35.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.356+0000 7f08b6ad8640 1 Processor -- start 2024-03-18T15:34:35.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.356+0000 7f08b6ad8640 1 -- start start 2024-03-18T15:34:35.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.356+0000 7f08b6ad8640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f08b0072b20 0x7f08b006d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:35.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.356+0000 7f08b6ad8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08b00b6520 0x7f08b00b9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:35.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.356+0000 7f08b6ad8640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f08b00b6c60 0x7f08b0139770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:35.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.356+0000 7f08b6ad8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f08b0152f50 con 0x7f08b00b6520 2024-03-18T15:34:35.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.356+0000 7f08b6ad8640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f08b0152dd0 con 0x7f08b00b6c60 2024-03-18T15:34:35.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.356+0000 7f08b6ad8640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f08b01530d0 con 0x7f08b0072b20 2024-03-18T15:34:35.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.356+0000 7f08affff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08b00b6520 0x7f08b00b9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:35.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.356+0000 7f08affff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08b00b6520 0x7f08b00b9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56562/0 (socket says 172.21.15.5:56562) 2024-03-18T15:34:35.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.356+0000 7f08affff640 1 -- 172.21.15.5:0/53092574 learned_addr learned my addr 172.21.15.5:0/53092574 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:35.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.356+0000 7f08b4874640 1 --2- 172.21.15.5:0/53092574 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f08b0072b20 0x7f08b006d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:35.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.356+0000 7f08affff640 1 -- 172.21.15.5:0/53092574 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f08b0072b20 msgr2=0x7f08b006d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:35.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.356+0000 7f08affff640 1 --2- 172.21.15.5:0/53092574 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f08b0072b20 0x7f08b006d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:35.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.356+0000 7f08b5075640 1 --2- 172.21.15.5:0/53092574 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f08b00b6c60 0x7f08b0139770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:35.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.356+0000 7f08affff640 1 -- 172.21.15.5:0/53092574 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f08b00b6c60 msgr2=0x7f08b0139770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:35.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.356+0000 7f08affff640 1 --2- 172.21.15.5:0/53092574 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f08b00b6c60 0x7f08b0139770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:35.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.356+0000 7f08affff640 1 -- 172.21.15.5:0/53092574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08b0139cb0 con 0x7f08b00b6520 2024-03-18T15:34:35.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.356+0000 7f08affff640 1 --2- 172.21.15.5:0/53092574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08b00b6520 0x7f08b00b9cf0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f08a40ac040 tx=0x7f08a401ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:35.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.356+0000 7f08ad7fa640 1 -- 172.21.15.5:0/53092574 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a40aada0 con 0x7f08b00b6520 2024-03-18T15:34:35.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.356+0000 7f08ad7fa640 1 -- 172.21.15.5:0/53092574 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08a40a6040 con 0x7f08b00b6520 2024-03-18T15:34:35.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.356+0000 7f08b6ad8640 1 -- 172.21.15.5:0/53092574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08b006c960 con 0x7f08b00b6520 2024-03-18T15:34:35.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.356+0000 7f08ad7fa640 1 -- 172.21.15.5:0/53092574 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a40bd630 con 0x7f08b00b6520 2024-03-18T15:34:35.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.356+0000 7f08b6ad8640 1 -- 172.21.15.5:0/53092574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08b006cf20 con 0x7f08b00b6520 2024-03-18T15:34:35.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.360+0000 7f08b6ad8640 1 -- 172.21.15.5:0/53092574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08b004f550 con 0x7f08b00b6520 2024-03-18T15:34:35.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.360+0000 7f08ad7fa640 1 -- 172.21.15.5:0/53092574 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f08a40b33f0 con 0x7f08b00b6520 2024-03-18T15:34:35.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.360+0000 7f08ad7fa640 1 --2- 172.21.15.5:0/53092574 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f089c067690 0x7f089c069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:35.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.360+0000 7f08b4874640 1 --2- 172.21.15.5:0/53092574 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f089c067690 0x7f089c069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:35.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.360+0000 7f08ad7fa640 1 -- 172.21.15.5:0/53092574 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7f08a411fa90 con 0x7f08b00b6520 2024-03-18T15:34:35.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.360+0000 7f08b4874640 1 --2- 172.21.15.5:0/53092574 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f089c067690 0x7f089c069b50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f08a000b930 tx=0x7f08a0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:35.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.364+0000 7f08ad7fa640 1 -- 172.21.15.5:0/53092574 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f08a40c3060 con 0x7f08b00b6520 2024-03-18T15:34:35.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.560+0000 7f08b6ad8640 1 -- 172.21.15.5:0/53092574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f08b00673a0 con 0x7f08b00b6520 2024-03-18T15:34:35.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.560+0000 7f08ad7fa640 1 -- 172.21.15.5:0/53092574 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+3528 (secure 0 0 0) 0x7f08a4016150 con 0x7f08b00b6520 2024-03-18T15:34:35.563 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:34:35.563 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4619,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/2921781388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":2921781388},{"type":"v1","addr":"172.21.15.80:6833","nonce":2921781388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-18T15:34:22.108585+0000","modified":"2024-03-18T15:34:25.110609+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4968},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4968":{"gid":4968,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6835/2235154713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2235154713},{"type":"v1","addr":"172.21.15.5:6835","nonce":2235154713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4968,"qdb_cluster":[4968]},"id":2}]} 2024-03-18T15:34:35.564 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 16 2024-03-18T15:34:35.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.564+0000 7f08b6ad8640 1 -- 172.21.15.5:0/53092574 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f089c067690 msgr2=0x7f089c069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:35.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.564+0000 7f08b6ad8640 1 --2- 172.21.15.5:0/53092574 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f089c067690 0x7f089c069b50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f08a000b930 tx=0x7f08a0008040 comp rx=0 tx=0).stop 2024-03-18T15:34:35.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.564+0000 7f08b6ad8640 1 -- 172.21.15.5:0/53092574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08b00b6520 msgr2=0x7f08b00b9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:35.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.564+0000 7f08b6ad8640 1 --2- 172.21.15.5:0/53092574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08b00b6520 0x7f08b00b9cf0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f08a40ac040 tx=0x7f08a401ad10 comp rx=0 tx=0).stop 2024-03-18T15:34:35.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.564+0000 7f08b6ad8640 1 -- 172.21.15.5:0/53092574 shutdown_connections 2024-03-18T15:34:35.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.564+0000 7f08b6ad8640 1 --2- 172.21.15.5:0/53092574 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f089c067690 0x7f089c069b50 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:35.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.564+0000 7f08b6ad8640 1 --2- 172.21.15.5:0/53092574 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f08b00b6c60 0x7f08b0139770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:35.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.564+0000 7f08b6ad8640 1 --2- 172.21.15.5:0/53092574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f08b00b6520 0x7f08b00b9cf0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:35.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.564+0000 7f08b6ad8640 1 --2- 172.21.15.5:0/53092574 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f08b0072b20 0x7f08b006d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:35.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.564+0000 7f08b6ad8640 1 -- 172.21.15.5:0/53092574 >> 172.21.15.5:0/53092574 conn(0x7f08b0066fc0 msgr2=0x7f08b0143270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:35.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.564+0000 7f08b6ad8640 1 -- 172.21.15.5:0/53092574 shutdown_connections 2024-03-18T15:34:35.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.564+0000 7f08b6ad8640 1 -- 172.21.15.5:0/53092574 wait complete. 2024-03-18T15:34:35.581 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:34:35.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.688+0000 7fdedc611640 1 Processor -- start 2024-03-18T15:34:35.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.688+0000 7fdedc611640 1 -- start start 2024-03-18T15:34:35.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.692+0000 7fdedc611640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fded40af790 0x7fded4072b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:35.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.692+0000 7fdedc611640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fded40b26a0 0x7fded40b2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:35.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.692+0000 7fdedc611640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fded40afb70 con 0x7fded406d070 2024-03-18T15:34:35.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.692+0000 7fdedc611640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fded40b3030 con 0x7fded40af790 2024-03-18T15:34:35.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.692+0000 7fdedc611640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fded40b4a20 con 0x7fded40b26a0 2024-03-18T15:34:35.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.692+0000 7fdeda3ad640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fded40af790 0x7fded4072b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:35.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.692+0000 7fdeda3ad640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fded40af790 0x7fded4072b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56574/0 (socket says 172.21.15.5:56574) 2024-03-18T15:34:35.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.692+0000 7fdeda3ad640 1 -- 172.21.15.5:0/864631561 learned_addr learned my addr 172.21.15.5:0/864631561 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:35.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.692+0000 7fdedabae640 1 --2- 172.21.15.5:0/864631561 >> v2:172.21.15.80:3300/0 conn(0x7fded40b26a0 0x7fded40b2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:35.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.692+0000 7fdeda3ad640 1 -- 172.21.15.5:0/864631561 >> v2:172.21.15.80:3300/0 conn(0x7fded40b26a0 msgr2=0x7fded40b2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:35.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.692+0000 7fdeda3ad640 1 --2- 172.21.15.5:0/864631561 >> v2:172.21.15.80:3300/0 conn(0x7fded40b26a0 0x7fded40b2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:35.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.692+0000 7fdeda3ad640 1 -- 172.21.15.5:0/864631561 >> v1:172.21.15.80:6789/0 conn(0x7fded406d070 legacy=0x7fded4147780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:35.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.692+0000 7fdeda3ad640 1 -- 172.21.15.5:0/864631561 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fded40b4d00 con 0x7fded40af790 2024-03-18T15:34:35.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.692+0000 7fdeda3ad640 1 --2- 172.21.15.5:0/864631561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded40af790 0x7fded4072b40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fded4072580 tx=0x7fdec801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59d453429dfcbd1c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:35.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.692+0000 7fded93ab640 1 -- 172.21.15.5:0/864631561 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec80b0070 con 0x7fded40af790 2024-03-18T15:34:35.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.692+0000 7fded93ab640 1 -- 172.21.15.5:0/864631561 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdec80ab600 con 0x7fded40af790 2024-03-18T15:34:35.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.692+0000 7fded93ab640 1 -- 172.21.15.5:0/864631561 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec80ab9d0 con 0x7fded40af790 2024-03-18T15:34:35.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.692+0000 7fdedc611640 1 -- 172.21.15.5:0/864631561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded40af790 msgr2=0x7fded4072b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:35.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.692+0000 7fdedc611640 1 --2- 172.21.15.5:0/864631561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded40af790 0x7fded4072b40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fded4072580 tx=0x7fdec801c450 comp rx=0 tx=0).stop 2024-03-18T15:34:35.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.692+0000 7fdedc611640 1 -- 172.21.15.5:0/864631561 shutdown_connections 2024-03-18T15:34:35.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.692+0000 7fdedc611640 1 --2- 172.21.15.5:0/864631561 >> v2:172.21.15.80:3300/0 conn(0x7fded40b26a0 0x7fded40b2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:35.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.692+0000 7fdedc611640 1 --2- 172.21.15.5:0/864631561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded40af790 0x7fded4072b40 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:35.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.692+0000 7fdedc611640 1 -- 172.21.15.5:0/864631561 >> 172.21.15.5:0/864631561 conn(0x7fded40b5670 msgr2=0x7fded40b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:35.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.692+0000 7fdedc611640 1 -- 172.21.15.5:0/864631561 shutdown_connections 2024-03-18T15:34:35.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.692+0000 7fdedc611640 1 -- 172.21.15.5:0/864631561 wait complete. 2024-03-18T15:34:35.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.696+0000 7fdedc611640 1 Processor -- start 2024-03-18T15:34:35.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.696+0000 7fdedc611640 1 -- start start 2024-03-18T15:34:35.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.696+0000 7fdedc611640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded406d070 0x7fded413dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:35.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.696+0000 7fdedc611640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fded40af790 0x7fded41df950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:35.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.696+0000 7fdedc611640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fded40b26a0 0x7fded4134de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:35.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.696+0000 7fdedc611640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fded41516a0 con 0x7fded406d070 2024-03-18T15:34:35.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.696+0000 7fdedc611640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fded4151520 con 0x7fded40b26a0 2024-03-18T15:34:35.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.696+0000 7fdedc611640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fded4151820 con 0x7fded40af790 2024-03-18T15:34:35.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.696+0000 7fdeda3ad640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded406d070 0x7fded413dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:35.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.696+0000 7fdeda3ad640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded406d070 0x7fded413dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56578/0 (socket says 172.21.15.5:56578) 2024-03-18T15:34:35.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.696+0000 7fdeda3ad640 1 -- 172.21.15.5:0/969315958 learned_addr learned my addr 172.21.15.5:0/969315958 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:35.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.696+0000 7fded9bac640 1 --2- 172.21.15.5:0/969315958 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fded40af790 0x7fded41df950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:35.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.696+0000 7fdedabae640 1 --2- 172.21.15.5:0/969315958 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fded40b26a0 0x7fded4134de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:35.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.696+0000 7fdeda3ad640 1 -- 172.21.15.5:0/969315958 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fded40af790 msgr2=0x7fded41df950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:35.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.696+0000 7fdeda3ad640 1 --2- 172.21.15.5:0/969315958 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fded40af790 0x7fded41df950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:35.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.696+0000 7fdeda3ad640 1 -- 172.21.15.5:0/969315958 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fded40b26a0 msgr2=0x7fded4134de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:35.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.696+0000 7fdeda3ad640 1 --2- 172.21.15.5:0/969315958 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fded40b26a0 0x7fded4134de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:35.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.696+0000 7fdeda3ad640 1 -- 172.21.15.5:0/969315958 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fded4146d70 con 0x7fded406d070 2024-03-18T15:34:35.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.696+0000 7fded9bac640 1 --2- 172.21.15.5:0/969315958 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fded40af790 0x7fded41df950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:35.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.696+0000 7fdedabae640 1 --2- 172.21.15.5:0/969315958 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fded40b26a0 0x7fded4134de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:35.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.696+0000 7fdeda3ad640 1 --2- 172.21.15.5:0/969315958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded406d070 0x7fded413dbc0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdec801a080 tx=0x7fdec80191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:35.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.696+0000 7fdec6ffd640 1 -- 172.21.15.5:0/969315958 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec80b9070 con 0x7fded406d070 2024-03-18T15:34:35.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.696+0000 7fdec6ffd640 1 -- 172.21.15.5:0/969315958 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdec80b4400 con 0x7fded406d070 2024-03-18T15:34:35.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.696+0000 7fdec6ffd640 1 -- 172.21.15.5:0/969315958 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec80b0040 con 0x7fded406d070 2024-03-18T15:34:35.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.696+0000 7fdedc611640 1 -- 172.21.15.5:0/969315958 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fded4146290 con 0x7fded406d070 2024-03-18T15:34:35.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.696+0000 7fdedc611640 1 -- 172.21.15.5:0/969315958 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fded4142730 con 0x7fded406d070 2024-03-18T15:34:35.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.696+0000 7fdedc611640 1 -- 172.21.15.5:0/969315958 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fded404f550 con 0x7fded406d070 2024-03-18T15:34:35.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.700+0000 7fdec6ffd640 1 -- 172.21.15.5:0/969315958 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fdec80b33a0 con 0x7fded406d070 2024-03-18T15:34:35.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.700+0000 7fdec6ffd640 1 --2- 172.21.15.5:0/969315958 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fdeb8067660 0x7fdeb8069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:35.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.700+0000 7fdec6ffd640 1 -- 172.21.15.5:0/969315958 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7fdec811eb00 con 0x7fded406d070 2024-03-18T15:34:35.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.700+0000 7fded9bac640 1 --2- 172.21.15.5:0/969315958 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fdeb8067660 0x7fdeb8069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:35.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.700+0000 7fded9bac640 1 --2- 172.21.15.5:0/969315958 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fdeb8067660 0x7fdeb8069b20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fded41458f0 tx=0x7fded0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:35.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.704+0000 7fdec6ffd640 1 -- 172.21.15.5:0/969315958 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fdec80160d0 con 0x7fded406d070 2024-03-18T15:34:35.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.864+0000 7fdedc611640 1 -- 172.21.15.5:0/969315958 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fded40adb50 con 0x7fded406d070 2024-03-18T15:34:35.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.864+0000 7fdec6ffd640 1 -- 172.21.15.5:0/969315958 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) ==== 74+0+16513 (secure 0 0 0) 0x7fdec80ef780 con 0x7fded406d070 2024-03-18T15:34:35.869 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:34:35.869 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":30,"fsid":"ef54fe03-5aa7-4eb2-9b1e-66c9cc345b94","created":"2024-03-18T15:33:18.086614+0000","modified":"2024-03-18T15:34:23.736672+0000","last_up_change":"2024-03-18T15:33:29.229911+0000","last_in_change":"2024-03-18T15:33:20.323840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:28.963186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-18T15:34:20.652674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-18T15:34:21.110594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df1c754b-af59-4d16-8919-fbb6386b3c48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6804","nonce":976264754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6808","nonce":976264754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6815","nonce":976264754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6811","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6813","nonce":976264754}]},"public_addr":"172.21.15.5:6804/976264754","cluster_addr":"172.21.15.5:6808/976264754","heartbeat_back_addr":"172.21.15.5:6815/976264754","heartbeat_front_addr":"172.21.15.5:6813/976264754","state":["exists","up"]},{"osd":1,"uuid":"b9a08b5e-8af4-4144-bdba-d69ed1ac627a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6801","nonce":3901761126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6803","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6805","nonce":3901761126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6812","nonce":3901761126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6807","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6809","nonce":3901761126}]},"public_addr":"172.21.15.5:6801/3901761126","cluster_addr":"172.21.15.5:6805/3901761126","heartbeat_back_addr":"172.21.15.5:6812/3901761126","heartbeat_front_addr":"172.21.15.5:6809/3901761126","state":["exists","up"]},{"osd":2,"uuid":"c3e8bb76-e382-439d-b380-62ef92f1255a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6817","nonce":502968448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6819","nonce":502968448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6823","nonce":502968448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6821","nonce":502968448}]},"public_addr":"172.21.15.5:6817/502968448","cluster_addr":"172.21.15.5:6819/502968448","heartbeat_back_addr":"172.21.15.5:6823/502968448","heartbeat_front_addr":"172.21.15.5:6821/502968448","state":["exists","up"]},{"osd":3,"uuid":"e8c94625-f2cd-4eba-b53d-2e7356e1c02a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6825","nonce":2425932735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6827","nonce":2425932735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6831","nonce":2425932735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6829","nonce":2425932735}]},"public_addr":"172.21.15.5:6825/2425932735","cluster_addr":"172.21.15.5:6827/2425932735","heartbeat_back_addr":"172.21.15.5:6831/2425932735","heartbeat_front_addr":"172.21.15.5:6829/2425932735","state":["exists","up"]},{"osd":4,"uuid":"0099dd52-6265-4d32-82e4-3ef16e73add0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6817","nonce":976514285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6819","nonce":976514285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6823","nonce":976514285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6821","nonce":976514285}]},"public_addr":"172.21.15.80:6817/976514285","cluster_addr":"172.21.15.80:6819/976514285","heartbeat_back_addr":"172.21.15.80:6823/976514285","heartbeat_front_addr":"172.21.15.80:6821/976514285","state":["exists","up"]},{"osd":5,"uuid":"ed10c84f-8c39-41cc-9b64-7c482a94a48e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6825","nonce":627153447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6827","nonce":627153447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6831","nonce":627153447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6829","nonce":627153447}]},"public_addr":"172.21.15.80:6825/627153447","cluster_addr":"172.21.15.80:6827/627153447","heartbeat_back_addr":"172.21.15.80:6831/627153447","heartbeat_front_addr":"172.21.15.80:6829/627153447","state":["exists","up"]},{"osd":6,"uuid":"c7a3538a-7048-4fba-8781-b440f03c25db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6803","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6805","nonce":488342527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6807","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6809","nonce":488342527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6815","nonce":488342527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6811","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6813","nonce":488342527}]},"public_addr":"172.21.15.80:6805/488342527","cluster_addr":"172.21.15.80:6809/488342527","heartbeat_back_addr":"172.21.15.80:6815/488342527","heartbeat_front_addr":"172.21.15.80:6813/488342527","state":["exists","up"]},{"osd":7,"uuid":"f9418104-3a97-4110-bc3a-1f8151d0f757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6801","nonce":982831666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6804","nonce":982831666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6812","nonce":982831666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6808","nonce":982831666}]},"public_addr":"172.21.15.80:6801/982831666","cluster_addr":"172.21.15.80:6804/982831666","heartbeat_back_addr":"172.21.15.80:6812/982831666","heartbeat_front_addr":"172.21.15.80:6808/982831666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.970828+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.858116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.987556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.864603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.756293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.591502+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:34:35.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.868+0000 7fdedc611640 1 -- 172.21.15.5:0/969315958 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fdeb8067660 msgr2=0x7fdeb8069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:35.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.868+0000 7fdedc611640 1 --2- 172.21.15.5:0/969315958 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fdeb8067660 0x7fdeb8069b20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fded41458f0 tx=0x7fded0009040 comp rx=0 tx=0).stop 2024-03-18T15:34:35.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.868+0000 7fdedc611640 1 -- 172.21.15.5:0/969315958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded406d070 msgr2=0x7fded413dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:35.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.868+0000 7fdedc611640 1 --2- 172.21.15.5:0/969315958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded406d070 0x7fded413dbc0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdec801a080 tx=0x7fdec80191f0 comp rx=0 tx=0).stop 2024-03-18T15:34:35.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.868+0000 7fdedc611640 1 -- 172.21.15.5:0/969315958 shutdown_connections 2024-03-18T15:34:35.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.868+0000 7fdedc611640 1 --2- 172.21.15.5:0/969315958 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fdeb8067660 0x7fdeb8069b20 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:35.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.868+0000 7fdedc611640 1 --2- 172.21.15.5:0/969315958 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fded40b26a0 0x7fded4134de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:35.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.868+0000 7fdedc611640 1 --2- 172.21.15.5:0/969315958 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fded40af790 0x7fded41df950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:35.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.868+0000 7fdedc611640 1 --2- 172.21.15.5:0/969315958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fded406d070 0x7fded413dbc0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:35.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.868+0000 7fdedc611640 1 -- 172.21.15.5:0/969315958 >> 172.21.15.5:0/969315958 conn(0x7fded40b5670 msgr2=0x7fded414d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:35.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.868+0000 7fdedc611640 1 -- 172.21.15.5:0/969315958 shutdown_connections 2024-03-18T15:34:35.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.868+0000 7fdedc611640 1 -- 172.21.15.5:0/969315958 wait complete. 2024-03-18T15:34:35.886 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-18T15:34:35.886 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:34:35.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.996+0000 7fd061e16640 1 Processor -- start 2024-03-18T15:34:35.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.996+0000 7fd061e16640 1 -- start start 2024-03-18T15:34:36.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.996+0000 7fd061e16640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd05c0744d0 0x7fd05c1478c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:36.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.996+0000 7fd061e16640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd05c0b4740 0x7fd05c141070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:36.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.996+0000 7fd061e16640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd05c0748b0 con 0x7fd05c0b5810 2024-03-18T15:34:36.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.996+0000 7fd061e16640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fd05c147f50 con 0x7fd05c0b4740 2024-03-18T15:34:36.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.996+0000 7fd061e16640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fd05c148230 con 0x7fd05c0744d0 2024-03-18T15:34:36.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.996+0000 7fd05affd640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd05c0b5810 0x7fd05c0b4030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44320/0 (socket says 172.21.15.5:44320) 2024-03-18T15:34:36.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.996+0000 7fd05affd640 1 -- 172.21.15.5:0/2230972780 learned_addr learned my addr 172.21.15.5:0/2230972780 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:36.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.996+0000 7fd05b7fe640 1 --2- 172.21.15.5:0/2230972780 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd05c0744d0 0x7fd05c1478c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:36.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.996+0000 7fd05bfff640 1 --2- 172.21.15.5:0/2230972780 >> v2:172.21.15.80:3300/0 conn(0x7fd05c0b4740 0x7fd05c141070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:36.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.996+0000 7fd05a7fc640 1 -- 172.21.15.5:0/2230972780 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2213599992 0 0) 0x7fd05c0748b0 con 0x7fd05c0b5810 2024-03-18T15:34:36.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.996+0000 7fd05a7fc640 1 -- 172.21.15.5:0/2230972780 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd044003530 con 0x7fd05c0b5810 2024-03-18T15:34:36.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.996+0000 7fd05a7fc640 1 -- 172.21.15.5:0/2230972780 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2981204069 0 0) 0x7fd044003530 con 0x7fd05c0b5810 2024-03-18T15:34:36.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.996+0000 7fd05a7fc640 1 -- 172.21.15.5:0/2230972780 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd05c0744d0 msgr2=0x7fd05c1478c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:36.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.996+0000 7fd05a7fc640 1 --2- 172.21.15.5:0/2230972780 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd05c0744d0 0x7fd05c1478c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:36.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.996+0000 7fd05a7fc640 1 -- 172.21.15.5:0/2230972780 >> v2:172.21.15.80:3300/0 conn(0x7fd05c0b4740 msgr2=0x7fd05c141070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:36.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.996+0000 7fd05a7fc640 1 --2- 172.21.15.5:0/2230972780 >> v2:172.21.15.80:3300/0 conn(0x7fd05c0b4740 0x7fd05c141070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:36.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.996+0000 7fd05a7fc640 1 -- 172.21.15.5:0/2230972780 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd05c0b3b10 con 0x7fd05c0b5810 2024-03-18T15:34:36.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.996+0000 7fd05a7fc640 1 -- 172.21.15.5:0/2230972780 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3837549003 0 0) 0x7fd050001800 con 0x7fd05c0b5810 2024-03-18T15:34:36.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.996+0000 7fd05a7fc640 1 -- 172.21.15.5:0/2230972780 >> v1:172.21.15.5:6789/0 conn(0x7fd05c0b5810 legacy=0x7fd05c0b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:36.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.996+0000 7fd05a7fc640 1 --2- 172.21.15.5:0/2230972780 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd05c0744d0 0x7fd044003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:36.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.996+0000 7fd05a7fc640 1 --2- 172.21.15.5:0/2230972780 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd044004020 0x7fd044004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:36.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.996+0000 7fd05a7fc640 1 --2- 172.21.15.5:0/2230972780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd044004d50 0x7fd0440051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:36.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.996+0000 7fd05a7fc640 1 -- 172.21.15.5:0/2230972780 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd05c147f50 con 0x7fd044004d50 2024-03-18T15:34:36.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.996+0000 7fd05a7fc640 1 -- 172.21.15.5:0/2230972780 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd05c148230 con 0x7fd044004020 2024-03-18T15:34:36.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.996+0000 7fd05a7fc640 1 -- 172.21.15.5:0/2230972780 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fd04400bb00 con 0x7fd05c0744d0 2024-03-18T15:34:36.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.996+0000 7fd05bfff640 1 --2- 172.21.15.5:0/2230972780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd044004d50 0x7fd0440051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:36.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.996+0000 7fd05b7fe640 1 --2- 172.21.15.5:0/2230972780 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd05c0744d0 0x7fd044003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:36.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.996+0000 7fd05affd640 1 --2- 172.21.15.5:0/2230972780 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd044004020 0x7fd044004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:36.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.996+0000 7fd05bfff640 1 -- 172.21.15.5:0/2230972780 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd05c0744d0 msgr2=0x7fd044003aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:36.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.996+0000 7fd05bfff640 1 --2- 172.21.15.5:0/2230972780 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd05c0744d0 0x7fd044003aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:36.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.996+0000 7fd05bfff640 1 -- 172.21.15.5:0/2230972780 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd044004020 msgr2=0x7fd044004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:36.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.996+0000 7fd05bfff640 1 --2- 172.21.15.5:0/2230972780 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd044004020 0x7fd044004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:36.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:35.996+0000 7fd05bfff640 1 -- 172.21.15.5:0/2230972780 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd044003530 con 0x7fd044004d50 2024-03-18T15:34:36.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.000+0000 7fd05bfff640 1 --2- 172.21.15.5:0/2230972780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd044004d50 0x7fd0440051d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd054009fc0 tx=0x7fd05401ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c3ef9d4840ca59c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:36.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.000+0000 7fd05a7fc640 1 -- 172.21.15.5:0/2230972780 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0540b2070 con 0x7fd044004d50 2024-03-18T15:34:36.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.000+0000 7fd05a7fc640 1 -- 172.21.15.5:0/2230972780 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0540a5ca0 con 0x7fd044004d50 2024-03-18T15:34:36.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.000+0000 7fd05a7fc640 1 -- 172.21.15.5:0/2230972780 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd054009c10 con 0x7fd044004d50 2024-03-18T15:34:36.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.000+0000 7fd061e16640 1 -- 172.21.15.5:0/2230972780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd044004d50 msgr2=0x7fd0440051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:36.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.000+0000 7fd061e16640 1 --2- 172.21.15.5:0/2230972780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd044004d50 0x7fd0440051d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd054009fc0 tx=0x7fd05401ccc0 comp rx=0 tx=0).stop 2024-03-18T15:34:36.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.000+0000 7fd05bfff640 1 -- 172.21.15.5:0/2230972780 reap_dead start 2024-03-18T15:34:36.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.000+0000 7fd061e16640 1 -- 172.21.15.5:0/2230972780 shutdown_connections 2024-03-18T15:34:36.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.000+0000 7fd061e16640 1 -- 172.21.15.5:0/2230972780 >> 172.21.15.5:0/2230972780 conn(0x7fd05c06bad0 msgr2=0x7fd05c0b9190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:36.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.000+0000 7fd061e16640 1 -- 172.21.15.5:0/2230972780 shutdown_connections 2024-03-18T15:34:36.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.000+0000 7fd061e16640 1 -- 172.21.15.5:0/2230972780 wait complete. 2024-03-18T15:34:36.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.000+0000 7fd061e16640 1 Processor -- start 2024-03-18T15:34:36.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.000+0000 7fd061e16640 1 -- start start 2024-03-18T15:34:36.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.000+0000 7fd061e16640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd05c138210 0x7fd05c0b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:36.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.000+0000 7fd061e16640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd05c1dc5c0 0x7fd05c143f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:36.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.000+0000 7fd061e16640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd05c1dca00 0x7fd05c144490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:36.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.000+0000 7fd061e16640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd05c147a40 con 0x7fd05c1dca00 2024-03-18T15:34:36.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.000+0000 7fd061e16640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd05c1478c0 con 0x7fd05c138210 2024-03-18T15:34:36.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.000+0000 7fd061e16640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fd05c147bc0 con 0x7fd05c1dc5c0 2024-03-18T15:34:36.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.000+0000 7fd05bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd05c1dca00 0x7fd05c144490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:36.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.000+0000 7fd05bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd05c1dca00 0x7fd05c144490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56596/0 (socket says 172.21.15.5:56596) 2024-03-18T15:34:36.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.000+0000 7fd05bfff640 1 -- 172.21.15.5:0/2042282140 learned_addr learned my addr 172.21.15.5:0/2042282140 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:36.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.000+0000 7fd05affd640 1 --2- 172.21.15.5:0/2042282140 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd05c1dc5c0 0x7fd05c143f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:36.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.000+0000 7fd05b7fe640 1 --2- 172.21.15.5:0/2042282140 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd05c138210 0x7fd05c0b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:36.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.000+0000 7fd05bfff640 1 -- 172.21.15.5:0/2042282140 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd05c1dc5c0 msgr2=0x7fd05c143f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:36.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.000+0000 7fd05bfff640 1 --2- 172.21.15.5:0/2042282140 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd05c1dc5c0 0x7fd05c143f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:36.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.000+0000 7fd05bfff640 1 -- 172.21.15.5:0/2042282140 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd05c138210 msgr2=0x7fd05c0b2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:36.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.000+0000 7fd05bfff640 1 --2- 172.21.15.5:0/2042282140 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd05c138210 0x7fd05c0b2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:36.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.000+0000 7fd05bfff640 1 -- 172.21.15.5:0/2042282140 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd05c1385f0 con 0x7fd05c1dca00 2024-03-18T15:34:36.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.004+0000 7fd05bfff640 1 --2- 172.21.15.5:0/2042282140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd05c1dca00 0x7fd05c144490 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd0540b0040 tx=0x7fd054009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:36.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.004+0000 7fd043fff640 1 -- 172.21.15.5:0/2042282140 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0540bd070 con 0x7fd05c1dca00 2024-03-18T15:34:36.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.004+0000 7fd043fff640 1 -- 172.21.15.5:0/2042282140 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0540a5e50 con 0x7fd05c1dca00 2024-03-18T15:34:36.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.004+0000 7fd043fff640 1 -- 172.21.15.5:0/2042282140 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0540b2040 con 0x7fd05c1dca00 2024-03-18T15:34:36.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.004+0000 7fd061e16640 1 -- 172.21.15.5:0/2042282140 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd05c1dfb30 con 0x7fd05c1dca00 2024-03-18T15:34:36.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.004+0000 7fd061e16640 1 -- 172.21.15.5:0/2042282140 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd05c1e00f0 con 0x7fd05c1dca00 2024-03-18T15:34:36.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.004+0000 7fd061e16640 1 -- 172.21.15.5:0/2042282140 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd05c04f550 con 0x7fd05c1dca00 2024-03-18T15:34:36.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.004+0000 7fd043fff640 1 -- 172.21.15.5:0/2042282140 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fd0540aeb60 con 0x7fd05c1dca00 2024-03-18T15:34:36.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.004+0000 7fd043fff640 1 --2- 172.21.15.5:0/2042282140 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd048067690 0x7fd048069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:36.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.004+0000 7fd043fff640 1 -- 172.21.15.5:0/2042282140 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7fd05411f4e0 con 0x7fd05c1dca00 2024-03-18T15:34:36.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.004+0000 7fd05b7fe640 1 --2- 172.21.15.5:0/2042282140 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd048067690 0x7fd048069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:36.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.004+0000 7fd05b7fe640 1 --2- 172.21.15.5:0/2042282140 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd048067690 0x7fd048069b50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd04c009a30 tx=0x7fd04c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:36.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.008+0000 7fd043fff640 1 -- 172.21.15.5:0/2042282140 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fd0540169d0 con 0x7fd05c1dca00 2024-03-18T15:34:36.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.204+0000 7fd061e16640 1 -- 172.21.15.5:0/2042282140 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd05c06c290 con 0x7fd05c1dca00 2024-03-18T15:34:36.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.204+0000 7fd043fff640 1 -- 172.21.15.5:0/2042282140 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+3528 (secure 0 0 0) 0x7fd0540efa80 con 0x7fd05c1dca00 2024-03-18T15:34:36.207 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:34:36.207 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4619,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/2921781388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":2921781388},{"type":"v1","addr":"172.21.15.80:6833","nonce":2921781388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-18T15:34:22.108585+0000","modified":"2024-03-18T15:34:25.110609+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4968},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4968":{"gid":4968,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6835/2235154713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2235154713},{"type":"v1","addr":"172.21.15.5:6835","nonce":2235154713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4968,"qdb_cluster":[4968]},"id":2}]} 2024-03-18T15:34:36.207 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 16 2024-03-18T15:34:36.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.204+0000 7fd061e16640 1 -- 172.21.15.5:0/2042282140 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd048067690 msgr2=0x7fd048069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:36.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.204+0000 7fd061e16640 1 --2- 172.21.15.5:0/2042282140 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd048067690 0x7fd048069b50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd04c009a30 tx=0x7fd04c008040 comp rx=0 tx=0).stop 2024-03-18T15:34:36.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.204+0000 7fd061e16640 1 -- 172.21.15.5:0/2042282140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd05c1dca00 msgr2=0x7fd05c144490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:36.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.204+0000 7fd061e16640 1 --2- 172.21.15.5:0/2042282140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd05c1dca00 0x7fd05c144490 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd0540b0040 tx=0x7fd054009750 comp rx=0 tx=0).stop 2024-03-18T15:34:36.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.204+0000 7fd061e16640 1 -- 172.21.15.5:0/2042282140 shutdown_connections 2024-03-18T15:34:36.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.204+0000 7fd061e16640 1 --2- 172.21.15.5:0/2042282140 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd048067690 0x7fd048069b50 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:36.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.204+0000 7fd061e16640 1 --2- 172.21.15.5:0/2042282140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd05c1dca00 0x7fd05c144490 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:36.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.204+0000 7fd061e16640 1 --2- 172.21.15.5:0/2042282140 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd05c1dc5c0 0x7fd05c143f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:36.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.204+0000 7fd061e16640 1 --2- 172.21.15.5:0/2042282140 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd05c138210 0x7fd05c0b2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:36.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.204+0000 7fd061e16640 1 -- 172.21.15.5:0/2042282140 >> 172.21.15.5:0/2042282140 conn(0x7fd05c06bad0 msgr2=0x7fd05c1396a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:36.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.204+0000 7fd061e16640 1 -- 172.21.15.5:0/2042282140 shutdown_connections 2024-03-18T15:34:36.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.204+0000 7fd061e16640 1 -- 172.21.15.5:0/2042282140 wait complete. 2024-03-18T15:34:36.226 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:34:36.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.340+0000 7fe76e321640 1 Processor -- start 2024-03-18T15:34:36.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.340+0000 7fe76e321640 1 -- start start 2024-03-18T15:34:36.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.340+0000 7fe76e321640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe7680b58a0 0x7fe7680b76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:36.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.340+0000 7fe76e321640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe76806dd20 0x7fe768147570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:36.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.340+0000 7fe76e321640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe768141210 0x7fe768141630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:36.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.340+0000 7fe76e321640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fe7680b7c10 con 0x7fe768141210 2024-03-18T15:34:36.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.340+0000 7fe76e321640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fe76806ce80 con 0x7fe76806dd20 2024-03-18T15:34:36.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.340+0000 7fe76e321640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fe768072c00 con 0x7fe7680b58a0 2024-03-18T15:34:36.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.340+0000 7fe76c8be640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe768141210 0x7fe768141630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:36.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.340+0000 7fe76c8be640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fe768141210 0x7fe768141630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56600/0 (socket says 172.21.15.5:56600) 2024-03-18T15:34:36.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.340+0000 7fe76c8be640 1 -- 172.21.15.5:0/2816260977 learned_addr learned my addr 172.21.15.5:0/2816260977 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:36.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.340+0000 7fe767fff640 1 --2- 172.21.15.5:0/2816260977 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe7680b58a0 0x7fe7680b76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:36.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.340+0000 7fe7677fe640 1 --2- 172.21.15.5:0/2816260977 >> v2:172.21.15.80:3300/0 conn(0x7fe76806dd20 0x7fe768147570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:36.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.340+0000 7fe76c8be640 1 -- 172.21.15.5:0/2816260977 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe7680b58a0 msgr2=0x7fe7680b76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:36.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.340+0000 7fe76c8be640 1 --2- 172.21.15.5:0/2816260977 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe7680b58a0 0x7fe7680b76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:36.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.340+0000 7fe76c8be640 1 -- 172.21.15.5:0/2816260977 >> v2:172.21.15.80:3300/0 conn(0x7fe76806dd20 msgr2=0x7fe768147570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:36.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.340+0000 7fe76c8be640 1 --2- 172.21.15.5:0/2816260977 >> v2:172.21.15.80:3300/0 conn(0x7fe76806dd20 0x7fe768147570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:36.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.340+0000 7fe76c8be640 1 -- 172.21.15.5:0/2816260977 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7680b5c80 con 0x7fe768141210 2024-03-18T15:34:36.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.344+0000 7fe76c8be640 1 --2- 172.21.15.5:0/2816260977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe768141210 0x7fe768141630 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe7680b8110 tx=0x7fe76001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=967fe55ef35d229 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:36.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.344+0000 7fe766ffd640 1 -- 172.21.15.5:0/2816260977 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7600a5670 con 0x7fe768141210 2024-03-18T15:34:36.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.344+0000 7fe766ffd640 1 -- 172.21.15.5:0/2816260977 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7600a5810 con 0x7fe768141210 2024-03-18T15:34:36.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.344+0000 7fe766ffd640 1 -- 172.21.15.5:0/2816260977 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7600a5be0 con 0x7fe768141210 2024-03-18T15:34:36.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.344+0000 7fe76e321640 1 -- 172.21.15.5:0/2816260977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe768141210 msgr2=0x7fe768141630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:36.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.344+0000 7fe76e321640 1 --2- 172.21.15.5:0/2816260977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe768141210 0x7fe768141630 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe7680b8110 tx=0x7fe76001c9f0 comp rx=0 tx=0).stop 2024-03-18T15:34:36.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.344+0000 7fe76e321640 1 -- 172.21.15.5:0/2816260977 shutdown_connections 2024-03-18T15:34:36.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.344+0000 7fe76e321640 1 --2- 172.21.15.5:0/2816260977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe768141210 0x7fe768141630 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:36.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.344+0000 7fe76e321640 1 --2- 172.21.15.5:0/2816260977 >> v2:172.21.15.80:3300/0 conn(0x7fe76806dd20 0x7fe768147570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:36.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.344+0000 7fe76e321640 1 --2- 172.21.15.5:0/2816260977 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe7680b58a0 0x7fe7680b76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:36.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.344+0000 7fe76e321640 1 -- 172.21.15.5:0/2816260977 >> 172.21.15.5:0/2816260977 conn(0x7fe768066fc0 msgr2=0x7fe7680673d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:36.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.344+0000 7fe76e321640 1 -- 172.21.15.5:0/2816260977 shutdown_connections 2024-03-18T15:34:36.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.344+0000 7fe76e321640 1 -- 172.21.15.5:0/2816260977 wait complete. 2024-03-18T15:34:36.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.344+0000 7fe76e321640 1 Processor -- start 2024-03-18T15:34:36.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.344+0000 7fe76e321640 1 -- start start 2024-03-18T15:34:36.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.344+0000 7fe76e321640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe76806dd20 0x7fe768143890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:36.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.344+0000 7fe76e321640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7680b58a0 0x7fe7680b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:36.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.344+0000 7fe76e321640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe768141210 0x7fe7681dfc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:36.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.344+0000 7fe76e321640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe768141cf0 con 0x7fe768141210 2024-03-18T15:34:36.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.344+0000 7fe76e321640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe768141b70 con 0x7fe7680b58a0 2024-03-18T15:34:36.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.344+0000 7fe76e321640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fe76814b2f0 con 0x7fe76806dd20 2024-03-18T15:34:36.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.344+0000 7fe76c8be640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe768141210 0x7fe7681dfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:36.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.344+0000 7fe76c8be640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe768141210 0x7fe7681dfc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56612/0 (socket says 172.21.15.5:56612) 2024-03-18T15:34:36.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.344+0000 7fe76c8be640 1 -- 172.21.15.5:0/3531683095 learned_addr learned my addr 172.21.15.5:0/3531683095 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:36.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.344+0000 7fe767fff640 1 --2- 172.21.15.5:0/3531683095 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe76806dd20 0x7fe768143890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:36.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.344+0000 7fe7677fe640 1 --2- 172.21.15.5:0/3531683095 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7680b58a0 0x7fe7680b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:36.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.344+0000 7fe76c8be640 1 -- 172.21.15.5:0/3531683095 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe76806dd20 msgr2=0x7fe768143890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:36.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.344+0000 7fe76c8be640 1 --2- 172.21.15.5:0/3531683095 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe76806dd20 0x7fe768143890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:36.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.344+0000 7fe76c8be640 1 -- 172.21.15.5:0/3531683095 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7680b58a0 msgr2=0x7fe7680b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:36.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.344+0000 7fe76c8be640 1 --2- 172.21.15.5:0/3531683095 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7680b58a0 0x7fe7680b12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:36.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.344+0000 7fe76c8be640 1 -- 172.21.15.5:0/3531683095 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe768144d50 con 0x7fe768141210 2024-03-18T15:34:36.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.344+0000 7fe7677fe640 1 --2- 172.21.15.5:0/3531683095 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7680b58a0 0x7fe7680b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:34:36.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.348+0000 7fe76c8be640 1 --2- 172.21.15.5:0/3531683095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe768141210 0x7fe7681dfc10 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fe7600af040 tx=0x7fe760019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:36.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.348+0000 7fe764ff9640 1 -- 172.21.15.5:0/3531683095 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7600bc070 con 0x7fe768141210 2024-03-18T15:34:36.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.348+0000 7fe764ff9640 1 -- 172.21.15.5:0/3531683095 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7600b8640 con 0x7fe768141210 2024-03-18T15:34:36.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.348+0000 7fe76e321640 1 -- 172.21.15.5:0/3531683095 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7681452a0 con 0x7fe768141210 2024-03-18T15:34:36.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.348+0000 7fe764ff9640 1 -- 172.21.15.5:0/3531683095 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe760009510 con 0x7fe768141210 2024-03-18T15:34:36.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.348+0000 7fe76e321640 1 -- 172.21.15.5:0/3531683095 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe768144fc0 con 0x7fe768141210 2024-03-18T15:34:36.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.348+0000 7fe76e321640 1 -- 172.21.15.5:0/3531683095 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7680678f0 con 0x7fe768141210 2024-03-18T15:34:36.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.348+0000 7fe764ff9640 1 -- 172.21.15.5:0/3531683095 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fe7600b89f0 con 0x7fe768141210 2024-03-18T15:34:36.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.348+0000 7fe764ff9640 1 --2- 172.21.15.5:0/3531683095 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe754067660 0x7fe754069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:36.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.348+0000 7fe764ff9640 1 -- 172.21.15.5:0/3531683095 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7fe760120580 con 0x7fe768141210 2024-03-18T15:34:36.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.348+0000 7fe767fff640 1 --2- 172.21.15.5:0/3531683095 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe754067660 0x7fe754069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:36.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.348+0000 7fe767fff640 1 --2- 172.21.15.5:0/3531683095 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe754067660 0x7fe754069b20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe7680bbc60 tx=0x7fe758008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:36.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.352+0000 7fe764ff9640 1 -- 172.21.15.5:0/3531683095 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fe7600165e0 con 0x7fe768141210 2024-03-18T15:34:36.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.548+0000 7fe76e321640 1 -- 172.21.15.5:0/3531683095 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe76804f550 con 0x7fe768141210 2024-03-18T15:34:36.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.548+0000 7fe764ff9640 1 -- 172.21.15.5:0/3531683095 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+3528 (secure 0 0 0) 0x7fe7600f0d60 con 0x7fe768141210 2024-03-18T15:34:36.551 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:34:36.551 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4619,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/2921781388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":2921781388},{"type":"v1","addr":"172.21.15.80:6833","nonce":2921781388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-18T15:34:22.108585+0000","modified":"2024-03-18T15:34:25.110609+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4968},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4968":{"gid":4968,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.5:6835/2235154713","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":2235154713},{"type":"v1","addr":"172.21.15.5:6835","nonce":2235154713}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4968,"qdb_cluster":[4968]},"id":2}]} 2024-03-18T15:34:36.551 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 16 2024-03-18T15:34:36.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.548+0000 7fe76e321640 1 -- 172.21.15.5:0/3531683095 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe754067660 msgr2=0x7fe754069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:36.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.548+0000 7fe76e321640 1 --2- 172.21.15.5:0/3531683095 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe754067660 0x7fe754069b20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe7680bbc60 tx=0x7fe758008040 comp rx=0 tx=0).stop 2024-03-18T15:34:36.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.548+0000 7fe76e321640 1 -- 172.21.15.5:0/3531683095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe768141210 msgr2=0x7fe7681dfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:36.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.548+0000 7fe76e321640 1 --2- 172.21.15.5:0/3531683095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe768141210 0x7fe7681dfc10 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fe7600af040 tx=0x7fe760019790 comp rx=0 tx=0).stop 2024-03-18T15:34:36.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.548+0000 7fe76e321640 1 -- 172.21.15.5:0/3531683095 shutdown_connections 2024-03-18T15:34:36.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.548+0000 7fe76e321640 1 --2- 172.21.15.5:0/3531683095 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fe754067660 0x7fe754069b20 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:36.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.548+0000 7fe76e321640 1 --2- 172.21.15.5:0/3531683095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe768141210 0x7fe7681dfc10 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:36.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.548+0000 7fe76e321640 1 --2- 172.21.15.5:0/3531683095 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7680b58a0 0x7fe7680b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:36.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.548+0000 7fe76e321640 1 --2- 172.21.15.5:0/3531683095 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fe76806dd20 0x7fe768143890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:36.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.548+0000 7fe76e321640 1 -- 172.21.15.5:0/3531683095 >> 172.21.15.5:0/3531683095 conn(0x7fe768066fc0 msgr2=0x7fe7680b68d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:36.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.548+0000 7fe76e321640 1 -- 172.21.15.5:0/3531683095 shutdown_connections 2024-03-18T15:34:36.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.548+0000 7fe76e321640 1 -- 172.21.15.5:0/3531683095 wait complete. 2024-03-18T15:34:36.568 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:34:36.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.680+0000 7fb708e56640 1 Processor -- start 2024-03-18T15:34:36.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.680+0000 7fb708e56640 1 -- start start 2024-03-18T15:34:36.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.680+0000 7fb708e56640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb704148060 0x7fb704153150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:36.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.680+0000 7fb708e56640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb70406b8c0 con 0x7fb70414b310 2024-03-18T15:34:36.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.680+0000 7fb708e56640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7040b78b0 con 0x7fb70406b4e0 2024-03-18T15:34:36.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.680+0000 7fb708e56640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fb7040b9930 con 0x7fb704148060 2024-03-18T15:34:36.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.680+0000 7fb701d9b640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fb70414b310 0x7fb704147950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44324/0 (socket says 172.21.15.5:44324) 2024-03-18T15:34:36.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.680+0000 7fb701d9b640 1 -- 172.21.15.5:0/204020184 learned_addr learned my addr 172.21.15.5:0/204020184 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:36.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.680+0000 7fb702d9d640 1 --2- 172.21.15.5:0/204020184 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb704148060 0x7fb704153150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:36.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.680+0000 7fb70159a640 1 -- 172.21.15.5:0/204020184 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2967391834 0 0) 0x7fb70406b8c0 con 0x7fb70414b310 2024-03-18T15:34:36.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.680+0000 7fb70159a640 1 -- 172.21.15.5:0/204020184 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb6f0003530 con 0x7fb70414b310 2024-03-18T15:34:36.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.680+0000 7fb70159a640 1 -- 172.21.15.5:0/204020184 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1493942710 0 0) 0x7fb6f0003530 con 0x7fb70414b310 2024-03-18T15:34:36.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.680+0000 7fb70159a640 1 -- 172.21.15.5:0/204020184 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb704148060 msgr2=0x7fb704153150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:36.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.680+0000 7fb70159a640 1 --2- 172.21.15.5:0/204020184 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb704148060 0x7fb704153150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:36.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.680+0000 7fb70159a640 1 -- 172.21.15.5:0/204020184 >> v1:172.21.15.80:6789/0 conn(0x7fb70406b4e0 legacy=0x7fb70414ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:36.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.680+0000 7fb70159a640 1 -- 172.21.15.5:0/204020184 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7040af810 con 0x7fb70414b310 2024-03-18T15:34:36.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.680+0000 7fb70159a640 1 -- 172.21.15.5:0/204020184 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3837549003 0 0) 0x7fb6ec001a00 con 0x7fb70414b310 2024-03-18T15:34:36.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.680+0000 7fb70159a640 1 -- 172.21.15.5:0/204020184 >> v1:172.21.15.5:6789/0 conn(0x7fb70414b310 legacy=0x7fb704147950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:36.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.680+0000 7fb70159a640 1 --2- 172.21.15.5:0/204020184 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb704148060 0x7fb6f0003ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:36.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.680+0000 7fb70159a640 1 --2- 172.21.15.5:0/204020184 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb6f0004050 0x7fb6f0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:36.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.680+0000 7fb70159a640 1 --2- 172.21.15.5:0/204020184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6f0004d50 0x7fb6f00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:36.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.680+0000 7fb70159a640 1 -- 172.21.15.5:0/204020184 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb7040b9930 con 0x7fb6f0004d50 2024-03-18T15:34:36.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.680+0000 7fb70159a640 1 -- 172.21.15.5:0/204020184 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f000bc30 con 0x7fb6f0004050 2024-03-18T15:34:36.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.680+0000 7fb70159a640 1 -- 172.21.15.5:0/204020184 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fb6f000be30 con 0x7fb704148060 2024-03-18T15:34:36.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.680+0000 7fb702d9d640 1 --2- 172.21.15.5:0/204020184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6f0004d50 0x7fb6f00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:36.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.680+0000 7fb701d9b640 1 --2- 172.21.15.5:0/204020184 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb6f0004050 0x7fb6f0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:36.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.680+0000 7fb70259c640 1 --2- 172.21.15.5:0/204020184 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb704148060 0x7fb6f0003ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:36.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.680+0000 7fb702d9d640 1 -- 172.21.15.5:0/204020184 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb704148060 msgr2=0x7fb6f0003ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:36.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.680+0000 7fb702d9d640 1 --2- 172.21.15.5:0/204020184 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb704148060 0x7fb6f0003ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:36.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.680+0000 7fb702d9d640 1 -- 172.21.15.5:0/204020184 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb6f0004050 msgr2=0x7fb6f0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:36.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.680+0000 7fb702d9d640 1 --2- 172.21.15.5:0/204020184 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb6f0004050 0x7fb6f0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:36.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.680+0000 7fb702d9d640 1 -- 172.21.15.5:0/204020184 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f0003530 con 0x7fb6f0004d50 2024-03-18T15:34:36.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.680+0000 7fb70259c640 1 --2- 172.21.15.5:0/204020184 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb704148060 0x7fb6f0003ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:36.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.684+0000 7fb702d9d640 1 --2- 172.21.15.5:0/204020184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6f0004d50 0x7fb6f00051d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4009ae0 tx=0x7fb6f401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=770ae971fe9e934e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:36.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.684+0000 7fb70159a640 1 -- 172.21.15.5:0/204020184 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f40b2070 con 0x7fb6f0004d50 2024-03-18T15:34:36.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.684+0000 7fb70159a640 1 -- 172.21.15.5:0/204020184 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6f40a5550 con 0x7fb6f0004d50 2024-03-18T15:34:36.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.684+0000 7fb70159a640 1 -- 172.21.15.5:0/204020184 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f40096b0 con 0x7fb6f0004d50 2024-03-18T15:34:36.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.684+0000 7fb708e56640 1 -- 172.21.15.5:0/204020184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6f0004d50 msgr2=0x7fb6f00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:36.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.684+0000 7fb708e56640 1 --2- 172.21.15.5:0/204020184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb6f0004d50 0x7fb6f00051d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4009ae0 tx=0x7fb6f401ca70 comp rx=0 tx=0).stop 2024-03-18T15:34:36.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.684+0000 7fb702d9d640 1 -- 172.21.15.5:0/204020184 reap_dead start 2024-03-18T15:34:36.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.684+0000 7fb708e56640 1 -- 172.21.15.5:0/204020184 shutdown_connections 2024-03-18T15:34:36.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.684+0000 7fb708e56640 1 -- 172.21.15.5:0/204020184 >> 172.21.15.5:0/204020184 conn(0x7fb70406e170 msgr2=0x7fb70406e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:36.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.684+0000 7fb708e56640 1 -- 172.21.15.5:0/204020184 shutdown_connections 2024-03-18T15:34:36.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.684+0000 7fb708e56640 1 -- 172.21.15.5:0/204020184 wait complete. 2024-03-18T15:34:36.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.688+0000 7fb708e56640 1 Processor -- start 2024-03-18T15:34:36.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.688+0000 7fb708e56640 1 -- start start 2024-03-18T15:34:36.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.688+0000 7fb708e56640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb70414a500 0x7fb7041408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:36.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.688+0000 7fb708e56640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb70414a8e0 0x7fb704140e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:36.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.688+0000 7fb708e56640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7040b13f0 0x7fb7041455e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:36.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.688+0000 7fb708e56640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb70409f5f0 con 0x7fb7040b13f0 2024-03-18T15:34:36.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.688+0000 7fb708e56640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb7040af190 con 0x7fb70414a500 2024-03-18T15:34:36.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.688+0000 7fb708e56640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fb70409f770 con 0x7fb70414a8e0 2024-03-18T15:34:36.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.688+0000 7fb702d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7040b13f0 0x7fb7041455e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:36.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.688+0000 7fb702d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7040b13f0 0x7fb7041455e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56624/0 (socket says 172.21.15.5:56624) 2024-03-18T15:34:36.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.688+0000 7fb702d9d640 1 -- 172.21.15.5:0/712917401 learned_addr learned my addr 172.21.15.5:0/712917401 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:36.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.688+0000 7fb701d9b640 1 --2- 172.21.15.5:0/712917401 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb70414a8e0 0x7fb704140e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:36.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.688+0000 7fb70259c640 1 --2- 172.21.15.5:0/712917401 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb70414a500 0x7fb7041408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:36.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.688+0000 7fb702d9d640 1 -- 172.21.15.5:0/712917401 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb70414a8e0 msgr2=0x7fb704140e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:36.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.688+0000 7fb702d9d640 1 --2- 172.21.15.5:0/712917401 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb70414a8e0 0x7fb704140e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:36.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.688+0000 7fb702d9d640 1 -- 172.21.15.5:0/712917401 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb70414a500 msgr2=0x7fb7041408e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:36.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.688+0000 7fb702d9d640 1 --2- 172.21.15.5:0/712917401 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb70414a500 0x7fb7041408e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:36.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.688+0000 7fb702d9d640 1 -- 172.21.15.5:0/712917401 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7040b1940 con 0x7fb7040b13f0 2024-03-18T15:34:36.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.688+0000 7fb702d9d640 1 --2- 172.21.15.5:0/712917401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7040b13f0 0x7fb7041455e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4009ae0 tx=0x7fb6f0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:36.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.688+0000 7fb6faffd640 1 -- 172.21.15.5:0/712917401 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f40b2040 con 0x7fb7040b13f0 2024-03-18T15:34:36.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.688+0000 7fb6faffd640 1 -- 172.21.15.5:0/712917401 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6f4000e50 con 0x7fb7040b13f0 2024-03-18T15:34:36.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.688+0000 7fb708e56640 1 -- 172.21.15.5:0/712917401 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb704149750 con 0x7fb7040b13f0 2024-03-18T15:34:36.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.688+0000 7fb6faffd640 1 -- 172.21.15.5:0/712917401 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f40096b0 con 0x7fb7040b13f0 2024-03-18T15:34:36.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.688+0000 7fb708e56640 1 -- 172.21.15.5:0/712917401 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb704149440 con 0x7fb7040b13f0 2024-03-18T15:34:36.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.688+0000 7fb708e56640 1 -- 172.21.15.5:0/712917401 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb70404f550 con 0x7fb7040b13f0 2024-03-18T15:34:36.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.692+0000 7fb6faffd640 1 -- 172.21.15.5:0/712917401 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fb6f40bd030 con 0x7fb7040b13f0 2024-03-18T15:34:36.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.692+0000 7fb6faffd640 1 --2- 172.21.15.5:0/712917401 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fb6e4067690 0x7fb6e4069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:36.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.692+0000 7fb6faffd640 1 -- 172.21.15.5:0/712917401 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7fb6f411e880 con 0x7fb7040b13f0 2024-03-18T15:34:36.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.692+0000 7fb70259c640 1 --2- 172.21.15.5:0/712917401 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fb6e4067690 0x7fb6e4069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:36.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.692+0000 7fb70259c640 1 --2- 172.21.15.5:0/712917401 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fb6e4067690 0x7fb6e4069b50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc000c10 tx=0x7fb6fc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:36.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.696+0000 7fb6faffd640 1 -- 172.21.15.5:0/712917401 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fb6f4016780 con 0x7fb7040b13f0 2024-03-18T15:34:36.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.856+0000 7fb708e56640 1 -- 172.21.15.5:0/712917401 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb70406ca50 con 0x7fb7040b13f0 2024-03-18T15:34:36.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.860+0000 7fb6faffd640 1 -- 172.21.15.5:0/712917401 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) ==== 74+0+16513 (secure 0 0 0) 0x7fb6f40a6e20 con 0x7fb7040b13f0 2024-03-18T15:34:36.862 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:34:36.862 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":30,"fsid":"ef54fe03-5aa7-4eb2-9b1e-66c9cc345b94","created":"2024-03-18T15:33:18.086614+0000","modified":"2024-03-18T15:34:23.736672+0000","last_up_change":"2024-03-18T15:33:29.229911+0000","last_in_change":"2024-03-18T15:33:20.323840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:28.963186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-18T15:34:20.652674+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-18T15:34:21.110594+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df1c754b-af59-4d16-8919-fbb6386b3c48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6804","nonce":976264754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6808","nonce":976264754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6815","nonce":976264754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6811","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6813","nonce":976264754}]},"public_addr":"172.21.15.5:6804/976264754","cluster_addr":"172.21.15.5:6808/976264754","heartbeat_back_addr":"172.21.15.5:6815/976264754","heartbeat_front_addr":"172.21.15.5:6813/976264754","state":["exists","up"]},{"osd":1,"uuid":"b9a08b5e-8af4-4144-bdba-d69ed1ac627a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6801","nonce":3901761126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6803","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6805","nonce":3901761126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6812","nonce":3901761126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6807","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6809","nonce":3901761126}]},"public_addr":"172.21.15.5:6801/3901761126","cluster_addr":"172.21.15.5:6805/3901761126","heartbeat_back_addr":"172.21.15.5:6812/3901761126","heartbeat_front_addr":"172.21.15.5:6809/3901761126","state":["exists","up"]},{"osd":2,"uuid":"c3e8bb76-e382-439d-b380-62ef92f1255a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6817","nonce":502968448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6819","nonce":502968448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6823","nonce":502968448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6821","nonce":502968448}]},"public_addr":"172.21.15.5:6817/502968448","cluster_addr":"172.21.15.5:6819/502968448","heartbeat_back_addr":"172.21.15.5:6823/502968448","heartbeat_front_addr":"172.21.15.5:6821/502968448","state":["exists","up"]},{"osd":3,"uuid":"e8c94625-f2cd-4eba-b53d-2e7356e1c02a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6825","nonce":2425932735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6827","nonce":2425932735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6831","nonce":2425932735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6829","nonce":2425932735}]},"public_addr":"172.21.15.5:6825/2425932735","cluster_addr":"172.21.15.5:6827/2425932735","heartbeat_back_addr":"172.21.15.5:6831/2425932735","heartbeat_front_addr":"172.21.15.5:6829/2425932735","state":["exists","up"]},{"osd":4,"uuid":"0099dd52-6265-4d32-82e4-3ef16e73add0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6817","nonce":976514285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6819","nonce":976514285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6823","nonce":976514285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6821","nonce":976514285}]},"public_addr":"172.21.15.80:6817/976514285","cluster_addr":"172.21.15.80:6819/976514285","heartbeat_back_addr":"172.21.15.80:6823/976514285","heartbeat_front_addr":"172.21.15.80:6821/976514285","state":["exists","up"]},{"osd":5,"uuid":"ed10c84f-8c39-41cc-9b64-7c482a94a48e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6825","nonce":627153447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6827","nonce":627153447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6831","nonce":627153447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6829","nonce":627153447}]},"public_addr":"172.21.15.80:6825/627153447","cluster_addr":"172.21.15.80:6827/627153447","heartbeat_back_addr":"172.21.15.80:6831/627153447","heartbeat_front_addr":"172.21.15.80:6829/627153447","state":["exists","up"]},{"osd":6,"uuid":"c7a3538a-7048-4fba-8781-b440f03c25db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6803","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6805","nonce":488342527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6807","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6809","nonce":488342527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6815","nonce":488342527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6811","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6813","nonce":488342527}]},"public_addr":"172.21.15.80:6805/488342527","cluster_addr":"172.21.15.80:6809/488342527","heartbeat_back_addr":"172.21.15.80:6815/488342527","heartbeat_front_addr":"172.21.15.80:6813/488342527","state":["exists","up"]},{"osd":7,"uuid":"f9418104-3a97-4110-bc3a-1f8151d0f757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6801","nonce":982831666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6804","nonce":982831666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6812","nonce":982831666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6808","nonce":982831666}]},"public_addr":"172.21.15.80:6801/982831666","cluster_addr":"172.21.15.80:6804/982831666","heartbeat_back_addr":"172.21.15.80:6812/982831666","heartbeat_front_addr":"172.21.15.80:6808/982831666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.970828+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.858116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.987556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.864603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.756293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.591502+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:34:36.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.860+0000 7fb708e56640 1 -- 172.21.15.5:0/712917401 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fb6e4067690 msgr2=0x7fb6e4069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:36.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.860+0000 7fb708e56640 1 --2- 172.21.15.5:0/712917401 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fb6e4067690 0x7fb6e4069b50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc000c10 tx=0x7fb6fc009040 comp rx=0 tx=0).stop 2024-03-18T15:34:36.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.860+0000 7fb708e56640 1 -- 172.21.15.5:0/712917401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7040b13f0 msgr2=0x7fb7041455e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:36.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.860+0000 7fb708e56640 1 --2- 172.21.15.5:0/712917401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7040b13f0 0x7fb7041455e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb6f4009ae0 tx=0x7fb6f0005840 comp rx=0 tx=0).stop 2024-03-18T15:34:36.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.860+0000 7fb708e56640 1 -- 172.21.15.5:0/712917401 shutdown_connections 2024-03-18T15:34:36.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.860+0000 7fb708e56640 1 --2- 172.21.15.5:0/712917401 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fb6e4067690 0x7fb6e4069b50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:36.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.860+0000 7fb708e56640 1 --2- 172.21.15.5:0/712917401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7040b13f0 0x7fb7041455e0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:36.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.860+0000 7fb708e56640 1 --2- 172.21.15.5:0/712917401 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb70414a8e0 0x7fb704140e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:36.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.860+0000 7fb708e56640 1 --2- 172.21.15.5:0/712917401 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb70414a500 0x7fb7041408e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:36.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.860+0000 7fb708e56640 1 -- 172.21.15.5:0/712917401 >> 172.21.15.5:0/712917401 conn(0x7fb70406e170 msgr2=0x7fb7040b9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:36.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.860+0000 7fb708e56640 1 -- 172.21.15.5:0/712917401 shutdown_connections 2024-03-18T15:34:36.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.860+0000 7fb708e56640 1 -- 172.21.15.5:0/712917401 wait complete. 2024-03-18T15:34:36.881 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-18T15:34:36.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.996+0000 7f04ea306640 1 Processor -- start 2024-03-18T15:34:36.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.996+0000 7f04ea306640 1 -- start start 2024-03-18T15:34:37.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.996+0000 7f04ea306640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f04e406b4e0 0x7f04e40702b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:37.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.996+0000 7f04ea306640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f04e4141360 0x7f04e41417e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:37.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.996+0000 7f04ea306640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04e406b8c0 con 0x7f04e41474c0 2024-03-18T15:34:37.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.996+0000 7f04ea306640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f04e406c0e0 con 0x7f04e406b4e0 2024-03-18T15:34:37.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.996+0000 7f04ea306640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f04e40719b0 con 0x7f04e4141360 2024-03-18T15:34:37.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.996+0000 7f04e88a3640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f04e41474c0 0x7f04e414aa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44334/0 (socket says 172.21.15.5:44334) 2024-03-18T15:34:37.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.996+0000 7f04e88a3640 1 -- 172.21.15.5:0/3517380800 learned_addr learned my addr 172.21.15.5:0/3517380800 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:37.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.996+0000 7f04e3fff640 1 --2- 172.21.15.5:0/3517380800 >> v2:172.21.15.80:3300/0 conn(0x7f04e406b4e0 0x7f04e40702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:37.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.996+0000 7f04e37fe640 1 --2- 172.21.15.5:0/3517380800 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f04e4141360 0x7f04e41417e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:37.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.996+0000 7f04e2ffd640 1 -- 172.21.15.5:0/3517380800 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3297693255 0 0) 0x7f04e406b8c0 con 0x7f04e41474c0 2024-03-18T15:34:37.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.996+0000 7f04e2ffd640 1 -- 172.21.15.5:0/3517380800 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f04d0003530 con 0x7f04e41474c0 2024-03-18T15:34:37.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.996+0000 7f04e2ffd640 1 -- 172.21.15.5:0/3517380800 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3999916643 0 0) 0x7f04d0003530 con 0x7f04e41474c0 2024-03-18T15:34:37.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.996+0000 7f04e2ffd640 1 -- 172.21.15.5:0/3517380800 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f04e4141360 msgr2=0x7f04e41417e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:37.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.996+0000 7f04e2ffd640 1 --2- 172.21.15.5:0/3517380800 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f04e4141360 0x7f04e41417e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:37.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.996+0000 7f04e2ffd640 1 -- 172.21.15.5:0/3517380800 >> v2:172.21.15.80:3300/0 conn(0x7f04e406b4e0 msgr2=0x7f04e40702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:37.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.996+0000 7f04e2ffd640 1 --2- 172.21.15.5:0/3517380800 >> v2:172.21.15.80:3300/0 conn(0x7f04e406b4e0 0x7f04e40702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:37.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.996+0000 7f04e2ffd640 1 -- 172.21.15.5:0/3517380800 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e406e800 con 0x7f04e41474c0 2024-03-18T15:34:37.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.996+0000 7f04e2ffd640 1 -- 172.21.15.5:0/3517380800 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3837549003 0 0) 0x7f04dc001680 con 0x7f04e41474c0 2024-03-18T15:34:37.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.996+0000 7f04e2ffd640 1 -- 172.21.15.5:0/3517380800 >> v1:172.21.15.5:6789/0 conn(0x7f04e41474c0 legacy=0x7f04e414aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:37.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.996+0000 7f04e2ffd640 1 --2- 172.21.15.5:0/3517380800 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f04e4141360 0x7f04d0003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:37.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.996+0000 7f04e2ffd640 1 --2- 172.21.15.5:0/3517380800 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04d0004020 0x7f04d0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:37.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.996+0000 7f04e2ffd640 1 --2- 172.21.15.5:0/3517380800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04d0004d50 0x7f04d00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:37.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.996+0000 7f04e2ffd640 1 -- 172.21.15.5:0/3517380800 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f04e406c0e0 con 0x7f04d0004d50 2024-03-18T15:34:37.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.996+0000 7f04e2ffd640 1 -- 172.21.15.5:0/3517380800 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f04e40719b0 con 0x7f04d0004020 2024-03-18T15:34:37.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.996+0000 7f04e2ffd640 1 -- 172.21.15.5:0/3517380800 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f04d000bb00 con 0x7f04e4141360 2024-03-18T15:34:37.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.996+0000 7f04e88a3640 1 --2- 172.21.15.5:0/3517380800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04d0004d50 0x7f04d00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:37.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.996+0000 7f04e3fff640 1 --2- 172.21.15.5:0/3517380800 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f04e4141360 0x7f04d0003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:37.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.996+0000 7f04e37fe640 1 --2- 172.21.15.5:0/3517380800 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04d0004020 0x7f04d0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:37.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.996+0000 7f04e88a3640 1 -- 172.21.15.5:0/3517380800 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f04e4141360 msgr2=0x7f04d0003aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:37.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.996+0000 7f04e88a3640 1 --2- 172.21.15.5:0/3517380800 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f04e4141360 0x7f04d0003aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:37.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.996+0000 7f04e88a3640 1 -- 172.21.15.5:0/3517380800 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04d0004020 msgr2=0x7f04d0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:37.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.996+0000 7f04e88a3640 1 --2- 172.21.15.5:0/3517380800 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04d0004020 0x7f04d0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:37.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:36.996+0000 7f04e88a3640 1 -- 172.21.15.5:0/3517380800 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04d0003530 con 0x7f04d0004d50 2024-03-18T15:34:37.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.000+0000 7f04e88a3640 1 --2- 172.21.15.5:0/3517380800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04d0004d50 0x7f04d00051d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f04dc00c210 tx=0x7f04dc01f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80bf51a79e18a086 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:37.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.000+0000 7f04e2ffd640 1 -- 172.21.15.5:0/3517380800 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04dc007d60 con 0x7f04d0004d50 2024-03-18T15:34:37.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.000+0000 7f04e2ffd640 1 -- 172.21.15.5:0/3517380800 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04dc005ae0 con 0x7f04d0004d50 2024-03-18T15:34:37.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.000+0000 7f04e2ffd640 1 -- 172.21.15.5:0/3517380800 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04dc007070 con 0x7f04d0004d50 2024-03-18T15:34:37.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.000+0000 7f04ea306640 1 -- 172.21.15.5:0/3517380800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04d0004d50 msgr2=0x7f04d00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:37.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.000+0000 7f04ea306640 1 --2- 172.21.15.5:0/3517380800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04d0004d50 0x7f04d00051d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f04dc00c210 tx=0x7f04dc01f3f0 comp rx=0 tx=0).stop 2024-03-18T15:34:37.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.000+0000 7f04e88a3640 1 -- 172.21.15.5:0/3517380800 reap_dead start 2024-03-18T15:34:37.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.000+0000 7f04ea306640 1 -- 172.21.15.5:0/3517380800 shutdown_connections 2024-03-18T15:34:37.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.000+0000 7f04ea306640 1 -- 172.21.15.5:0/3517380800 >> 172.21.15.5:0/3517380800 conn(0x7f04e40b7a80 msgr2=0x7f04e40b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:37.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.000+0000 7f04ea306640 1 -- 172.21.15.5:0/3517380800 shutdown_connections 2024-03-18T15:34:37.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.000+0000 7f04ea306640 1 -- 172.21.15.5:0/3517380800 wait complete. 2024-03-18T15:34:37.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.000+0000 7f04ea306640 1 Processor -- start 2024-03-18T15:34:37.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.000+0000 7f04ea306640 1 -- start start 2024-03-18T15:34:37.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.000+0000 7f04ea306640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04d0004d50 0x7f04e41d0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:37.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.000+0000 7f04ea306640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f04e41466a0 0x7f04e41d0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:37.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.000+0000 7f04ea306640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04e41dcc50 0x7f04e41441a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:37.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.000+0000 7f04ea306640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f04e41417e0 con 0x7f04e41dcc50 2024-03-18T15:34:37.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.000+0000 7f04ea306640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f04e4070910 con 0x7f04d0004d50 2024-03-18T15:34:37.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.000+0000 7f04ea306640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f04e4141960 con 0x7f04e41466a0 2024-03-18T15:34:37.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.004+0000 7f04e88a3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04e41dcc50 0x7f04e41441a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:37.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.004+0000 7f04e88a3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04e41dcc50 0x7f04e41441a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56644/0 (socket says 172.21.15.5:56644) 2024-03-18T15:34:37.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.004+0000 7f04e88a3640 1 -- 172.21.15.5:0/3006064288 learned_addr learned my addr 172.21.15.5:0/3006064288 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:37.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.004+0000 7f04e3fff640 1 --2- 172.21.15.5:0/3006064288 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04d0004d50 0x7f04e41d0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:37.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.004+0000 7f04e37fe640 1 --2- 172.21.15.5:0/3006064288 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f04e41466a0 0x7f04e41d0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:37.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.004+0000 7f04e88a3640 1 -- 172.21.15.5:0/3006064288 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f04e41466a0 msgr2=0x7f04e41d0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:37.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.004+0000 7f04e88a3640 1 --2- 172.21.15.5:0/3006064288 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f04e41466a0 0x7f04e41d0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:37.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.004+0000 7f04e88a3640 1 -- 172.21.15.5:0/3006064288 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04d0004d50 msgr2=0x7f04e41d0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:37.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.004+0000 7f04e88a3640 1 --2- 172.21.15.5:0/3006064288 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04d0004d50 0x7f04e41d0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:37.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.004+0000 7f04e88a3640 1 -- 172.21.15.5:0/3006064288 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e41d1f60 con 0x7f04e41dcc50 2024-03-18T15:34:37.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.004+0000 7f04e88a3640 1 --2- 172.21.15.5:0/3006064288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04e41dcc50 0x7f04e41441a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f04dc01fc90 tx=0x7f04d0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:37.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.004+0000 7f04e0ff9640 1 -- 172.21.15.5:0/3006064288 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04dc007d60 con 0x7f04e41dcc50 2024-03-18T15:34:37.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.004+0000 7f04e0ff9640 1 -- 172.21.15.5:0/3006064288 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04dc005c90 con 0x7f04e41dcc50 2024-03-18T15:34:37.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.004+0000 7f04ea306640 1 -- 172.21.15.5:0/3006064288 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04e40b5520 con 0x7f04e41dcc50 2024-03-18T15:34:37.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.004+0000 7f04e0ff9640 1 -- 172.21.15.5:0/3006064288 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04dc001cc0 con 0x7f04e41dcc50 2024-03-18T15:34:37.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.004+0000 7f04ea306640 1 -- 172.21.15.5:0/3006064288 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04e40b6310 con 0x7f04e41dcc50 2024-03-18T15:34:37.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.004+0000 7f04ea306640 1 -- 172.21.15.5:0/3006064288 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04e406c310 con 0x7f04e41dcc50 2024-03-18T15:34:37.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.004+0000 7f04e0ff9640 1 -- 172.21.15.5:0/3006064288 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f04dc0ba030 con 0x7f04e41dcc50 2024-03-18T15:34:37.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.004+0000 7f04e0ff9640 1 --2- 172.21.15.5:0/3006064288 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f04c4067660 0x7f04c4069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:37.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.004+0000 7f04e0ff9640 1 -- 172.21.15.5:0/3006064288 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5709+0+0 (secure 0 0 0) 0x7f04dc121fb0 con 0x7f04e41dcc50 2024-03-18T15:34:37.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.004+0000 7f04e3fff640 1 --2- 172.21.15.5:0/3006064288 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f04c4067660 0x7f04c4069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:37.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.004+0000 7f04e3fff640 1 --2- 172.21.15.5:0/3006064288 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f04c4067660 0x7f04c4069b20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f04d4007520 tx=0x7f04d400b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:37.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.008+0000 7f04e0ff9640 1 -- 172.21.15.5:0/3006064288 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f04dc0c1030 con 0x7f04e41dcc50 2024-03-18T15:34:37.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.196+0000 7f04ea306640 1 -- 172.21.15.5:0/3006064288 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f04e404f550 con 0x7f04e41dcc50 2024-03-18T15:34:37.201 INFO:tasks.ceph.mon.a.smithi005.stderr:2024-03-18T15:34:37.200+0000 7f3da0624640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-18T15:34:37.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.200+0000 7f04e0ff9640 1 -- 172.21.15.5:0/3006064288 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) ==== 161+0+0 (secure 0 0 0) 0x7f04dc019100 con 0x7f04e41dcc50 2024-03-18T15:34:37.203 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-18T15:34:37.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.204+0000 7f04ea306640 1 -- 172.21.15.5:0/3006064288 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f04c4067660 msgr2=0x7f04c4069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:37.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.204+0000 7f04ea306640 1 --2- 172.21.15.5:0/3006064288 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f04c4067660 0x7f04c4069b20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f04d4007520 tx=0x7f04d400b040 comp rx=0 tx=0).stop 2024-03-18T15:34:37.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.204+0000 7f04ea306640 1 -- 172.21.15.5:0/3006064288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04e41dcc50 msgr2=0x7f04e41441a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:37.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.204+0000 7f04ea306640 1 --2- 172.21.15.5:0/3006064288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04e41dcc50 0x7f04e41441a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f04dc01fc90 tx=0x7f04d0005840 comp rx=0 tx=0).stop 2024-03-18T15:34:37.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.204+0000 7f04ea306640 1 -- 172.21.15.5:0/3006064288 shutdown_connections 2024-03-18T15:34:37.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.204+0000 7f04ea306640 1 --2- 172.21.15.5:0/3006064288 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f04c4067660 0x7f04c4069b20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:37.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.204+0000 7f04ea306640 1 --2- 172.21.15.5:0/3006064288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04e41dcc50 0x7f04e41441a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:37.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.204+0000 7f04ea306640 1 --2- 172.21.15.5:0/3006064288 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f04e41466a0 0x7f04e41d0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:37.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.204+0000 7f04ea306640 1 --2- 172.21.15.5:0/3006064288 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04d0004d50 0x7f04e41d0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:37.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.204+0000 7f04ea306640 1 -- 172.21.15.5:0/3006064288 >> 172.21.15.5:0/3006064288 conn(0x7f04e40b7a80 msgr2=0x7f04e4140a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:37.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.204+0000 7f04ea306640 1 -- 172.21.15.5:0/3006064288 shutdown_connections 2024-03-18T15:34:37.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.204+0000 7f04ea306640 1 -- 172.21.15.5:0/3006064288 wait complete. 2024-03-18T15:34:37.211 INFO:tasks.ceph.mds.a.smithi005.stderr: -2448> 2024-03-18T15:34:26.688+0000 7fb984838640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-18T15:34:37.223 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-18T15:34:37.246 INFO:tasks.ceph.mds.a.smithi005.stdout:starting mds.a at 2024-03-18T15:34:37.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.348+0000 7f8c55322640 1 Processor -- start 2024-03-18T15:34:37.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.348+0000 7f8c55322640 1 -- start start 2024-03-18T15:34:37.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.348+0000 7f8c55322640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8c5006d2d0 0x7f8c500b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:37.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.348+0000 7f8c55322640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8c5013f260 0x7f8c5013f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:37.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.348+0000 7f8c55322640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c500b0ea0 con 0x7f8c501453a0 2024-03-18T15:34:37.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.348+0000 7f8c55322640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f8c500728c0 con 0x7f8c5013f260 2024-03-18T15:34:37.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.348+0000 7f8c55322640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f8c5006d6b0 con 0x7f8c5006d2d0 2024-03-18T15:34:37.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.348+0000 7f8c4e7fc640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f8c5013f260 0x7f8c5013f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:37.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.348+0000 7f8c4f7fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f8c501453a0 0x7f8c50145810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44350/0 (socket says 172.21.15.5:44350) 2024-03-18T15:34:37.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.348+0000 7f8c4f7fe640 1 -- 172.21.15.5:0/354337491 learned_addr learned my addr 172.21.15.5:0/354337491 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:37.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.348+0000 7f8c4effd640 1 --2- 172.21.15.5:0/354337491 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8c5006d2d0 0x7f8c500b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:37.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.348+0000 7f8c4dffb640 1 -- 172.21.15.5:0/354337491 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 664320998 0 0) 0x7f8c500b0ea0 con 0x7f8c501453a0 2024-03-18T15:34:37.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.348+0000 7f8c4dffb640 1 -- 172.21.15.5:0/354337491 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8c3c003530 con 0x7f8c501453a0 2024-03-18T15:34:37.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.348+0000 7f8c4e7fc640 1 -- 172.21.15.5:0/354337491 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8c5006d2d0 msgr2=0x7f8c500b23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:37.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.348+0000 7f8c4e7fc640 1 --2- 172.21.15.5:0/354337491 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8c5006d2d0 0x7f8c500b23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:37.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.348+0000 7f8c4e7fc640 1 -- 172.21.15.5:0/354337491 >> v1:172.21.15.5:6789/0 conn(0x7f8c501453a0 legacy=0x7f8c50145810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:37.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.348+0000 7f8c4e7fc640 1 -- 172.21.15.5:0/354337491 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c500b0bc0 con 0x7f8c5013f260 2024-03-18T15:34:37.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.352+0000 7f8c4e7fc640 1 --2- 172.21.15.5:0/354337491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c5013f260 0x7f8c5013f6a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8c40009d90 tx=0x7f8c4001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6078359e663104de server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:37.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.352+0000 7f8c4dffb640 1 -- 172.21.15.5:0/354337491 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c400b2070 con 0x7f8c5013f260 2024-03-18T15:34:37.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.352+0000 7f8c4dffb640 1 -- 172.21.15.5:0/354337491 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c400a58b0 con 0x7f8c5013f260 2024-03-18T15:34:37.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.352+0000 7f8c4dffb640 1 -- 172.21.15.5:0/354337491 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c400a5c80 con 0x7f8c5013f260 2024-03-18T15:34:37.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.352+0000 7f8c55322640 1 -- 172.21.15.5:0/354337491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c5013f260 msgr2=0x7f8c5013f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:37.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.352+0000 7f8c55322640 1 --2- 172.21.15.5:0/354337491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c5013f260 0x7f8c5013f6a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8c40009d90 tx=0x7f8c4001c8e0 comp rx=0 tx=0).stop 2024-03-18T15:34:37.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.352+0000 7f8c55322640 1 -- 172.21.15.5:0/354337491 shutdown_connections 2024-03-18T15:34:37.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.352+0000 7f8c55322640 1 --2- 172.21.15.5:0/354337491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c5013f260 0x7f8c5013f6a0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:37.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.352+0000 7f8c55322640 1 --2- 172.21.15.5:0/354337491 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8c5006d2d0 0x7f8c500b23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:37.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.352+0000 7f8c55322640 1 -- 172.21.15.5:0/354337491 >> 172.21.15.5:0/354337491 conn(0x7f8c500b5670 msgr2=0x7f8c500b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:37.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.352+0000 7f8c55322640 1 -- 172.21.15.5:0/354337491 shutdown_connections 2024-03-18T15:34:37.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.352+0000 7f8c55322640 1 -- 172.21.15.5:0/354337491 wait complete. 2024-03-18T15:34:37.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.352+0000 7f8c55322640 1 Processor -- start 2024-03-18T15:34:37.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.352+0000 7f8c55322640 1 -- start start 2024-03-18T15:34:37.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.356+0000 7f8c55322640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c5006d2d0 0x7f8c50071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:37.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.356+0000 7f8c55322640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8c5013f260 0x7f8c50142050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:37.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.356+0000 7f8c55322640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8c501453a0 0x7f8c5013fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:37.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.356+0000 7f8c55322640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8c500b52c0 con 0x7f8c5006d2d0 2024-03-18T15:34:37.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.356+0000 7f8c55322640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8c500b5140 con 0x7f8c5013f260 2024-03-18T15:34:37.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.356+0000 7f8c55322640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f8c50072d30 con 0x7f8c501453a0 2024-03-18T15:34:37.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.356+0000 7f8c4effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c5006d2d0 0x7f8c50071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:37.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.356+0000 7f8c4effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c5006d2d0 0x7f8c50071c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56674/0 (socket says 172.21.15.5:56674) 2024-03-18T15:34:37.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.356+0000 7f8c4effd640 1 -- 172.21.15.5:0/1114979431 learned_addr learned my addr 172.21.15.5:0/1114979431 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:37.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.356+0000 7f8c4f7fe640 1 --2- 172.21.15.5:0/1114979431 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8c501453a0 0x7f8c5013fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:37.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.356+0000 7f8c4e7fc640 1 --2- 172.21.15.5:0/1114979431 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8c5013f260 0x7f8c50142050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:37.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.356+0000 7f8c4effd640 1 -- 172.21.15.5:0/1114979431 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8c501453a0 msgr2=0x7f8c5013fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:37.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.356+0000 7f8c4effd640 1 --2- 172.21.15.5:0/1114979431 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8c501453a0 0x7f8c5013fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:37.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.356+0000 7f8c4effd640 1 -- 172.21.15.5:0/1114979431 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8c5013f260 msgr2=0x7f8c50142050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:37.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.356+0000 7f8c4effd640 1 --2- 172.21.15.5:0/1114979431 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8c5013f260 0x7f8c50142050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:37.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.356+0000 7f8c4effd640 1 -- 172.21.15.5:0/1114979431 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c500b33a0 con 0x7f8c5006d2d0 2024-03-18T15:34:37.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.356+0000 7f8c4effd640 1 --2- 172.21.15.5:0/1114979431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c5006d2d0 0x7f8c50071c00 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8c3800d620 tx=0x7f8c3800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:37.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.356+0000 7f8c477fe640 1 -- 172.21.15.5:0/1114979431 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3800b620 con 0x7f8c5006d2d0 2024-03-18T15:34:37.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.356+0000 7f8c477fe640 1 -- 172.21.15.5:0/1114979431 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c38019070 con 0x7f8c5006d2d0 2024-03-18T15:34:37.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.356+0000 7f8c55322640 1 -- 172.21.15.5:0/1114979431 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c5009e8a0 con 0x7f8c5006d2d0 2024-03-18T15:34:37.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.356+0000 7f8c477fe640 1 -- 172.21.15.5:0/1114979431 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3800ba20 con 0x7f8c5006d2d0 2024-03-18T15:34:37.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.356+0000 7f8c55322640 1 -- 172.21.15.5:0/1114979431 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c500aef80 con 0x7f8c5006d2d0 2024-03-18T15:34:37.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.356+0000 7f8c55322640 1 -- 172.21.15.5:0/1114979431 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c5004f550 con 0x7f8c5006d2d0 2024-03-18T15:34:37.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.356+0000 7f8c477fe640 1 -- 172.21.15.5:0/1114979431 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f8c3800bbc0 con 0x7f8c5006d2d0 2024-03-18T15:34:37.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.356+0000 7f8c477fe640 1 --2- 172.21.15.5:0/1114979431 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f8c30067660 0x7f8c30069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:37.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.356+0000 7f8c4e7fc640 1 --2- 172.21.15.5:0/1114979431 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f8c30067660 0x7f8c30069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:37.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.356+0000 7f8c477fe640 1 -- 172.21.15.5:0/1114979431 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5795+0+0 (secure 0 0 0) 0x7f8c38085d00 con 0x7f8c5006d2d0 2024-03-18T15:34:37.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.356+0000 7f8c4e7fc640 1 --2- 172.21.15.5:0/1114979431 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f8c30067660 0x7f8c30069b20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8c400af000 tx=0x7f8c400a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:37.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.360+0000 7f8c477fe640 1 -- 172.21.15.5:0/1114979431 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f8c380569b0 con 0x7f8c5006d2d0 2024-03-18T15:34:37.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:37.552+0000 7f8c55322640 1 -- 172.21.15.5:0/1114979431 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f8c50143720 con 0x7f8c5006d2d0 2024-03-18T15:34:38.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.204+0000 7f8c477fe640 1 -- 172.21.15.5:0/1114979431 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) ==== 104+0+0 (secure 0 0 0) 0x7f8c380562b0 con 0x7f8c5006d2d0 2024-03-18T15:34:38.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.204+0000 7f8c55322640 1 -- 172.21.15.5:0/1114979431 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f8c30067660 msgr2=0x7f8c30069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:38.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.204+0000 7f8c55322640 1 --2- 172.21.15.5:0/1114979431 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f8c30067660 0x7f8c30069b20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8c400af000 tx=0x7f8c400a7070 comp rx=0 tx=0).stop 2024-03-18T15:34:38.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.204+0000 7f8c55322640 1 -- 172.21.15.5:0/1114979431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c5006d2d0 msgr2=0x7f8c50071c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:38.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.204+0000 7f8c55322640 1 --2- 172.21.15.5:0/1114979431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c5006d2d0 0x7f8c50071c00 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8c3800d620 tx=0x7f8c3800dae0 comp rx=0 tx=0).stop 2024-03-18T15:34:38.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.204+0000 7f8c55322640 1 -- 172.21.15.5:0/1114979431 shutdown_connections 2024-03-18T15:34:38.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.204+0000 7f8c55322640 1 --2- 172.21.15.5:0/1114979431 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f8c30067660 0x7f8c30069b20 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:38.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.204+0000 7f8c55322640 1 --2- 172.21.15.5:0/1114979431 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8c501453a0 0x7f8c5013fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:38.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.204+0000 7f8c55322640 1 --2- 172.21.15.5:0/1114979431 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8c5013f260 0x7f8c50142050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:38.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.204+0000 7f8c55322640 1 --2- 172.21.15.5:0/1114979431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c5006d2d0 0x7f8c50071c00 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:38.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.204+0000 7f8c55322640 1 -- 172.21.15.5:0/1114979431 >> 172.21.15.5:0/1114979431 conn(0x7f8c500b5670 msgr2=0x7f8c50143c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:38.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.208+0000 7f8c55322640 1 -- 172.21.15.5:0/1114979431 shutdown_connections 2024-03-18T15:34:38.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.208+0000 7f8c55322640 1 -- 172.21.15.5:0/1114979431 wait complete. 2024-03-18T15:34:38.231 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-18T15:34:38.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.404+0000 7fd0bd47b640 1 Processor -- start 2024-03-18T15:34:38.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.404+0000 7fd0bd47b640 1 -- start start 2024-03-18T15:34:38.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.404+0000 7fd0bd47b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd0b809f0f0 0x7fd0b80726a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:38.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.404+0000 7fd0bd47b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd0b806d650 0x7fd0b8072be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:38.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.404+0000 7fd0bd47b640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0b80b5bd0 con 0x7fd0b80b57f0 2024-03-18T15:34:38.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.404+0000 7fd0bd47b640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fd0b806ddf0 con 0x7fd0b809f0f0 2024-03-18T15:34:38.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.404+0000 7fd0bd47b640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fd0b806e430 con 0x7fd0b806d650 2024-03-18T15:34:38.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.404+0000 7fd0b67fc640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd0b809f0f0 0x7fd0b80726a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:38.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.404+0000 7fd0b6ffd640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd0b80b57f0 0x7fd0b8145250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44366/0 (socket says 172.21.15.5:44366) 2024-03-18T15:34:38.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.404+0000 7fd0b6ffd640 1 -- 172.21.15.5:0/1084005639 learned_addr learned my addr 172.21.15.5:0/1084005639 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:38.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.404+0000 7fd0b77fe640 1 --2- 172.21.15.5:0/1084005639 >> v2:172.21.15.80:3300/0 conn(0x7fd0b806d650 0x7fd0b8072be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:38.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.404+0000 7fd0b67fc640 1 -- 172.21.15.5:0/1084005639 >> v2:172.21.15.80:3300/0 conn(0x7fd0b806d650 msgr2=0x7fd0b8072be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:38.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.404+0000 7fd0b67fc640 1 --2- 172.21.15.5:0/1084005639 >> v2:172.21.15.80:3300/0 conn(0x7fd0b806d650 0x7fd0b8072be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:38.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.404+0000 7fd0b67fc640 1 -- 172.21.15.5:0/1084005639 >> v1:172.21.15.5:6789/0 conn(0x7fd0b80b57f0 legacy=0x7fd0b8145250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:38.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.404+0000 7fd0b67fc640 1 -- 172.21.15.5:0/1084005639 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0b80b6410 con 0x7fd0b809f0f0 2024-03-18T15:34:38.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.404+0000 7fd0b5ffb640 1 -- 172.21.15.5:0/1084005639 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4234411151 0 0) 0x7fd0b80b5bd0 con 0x7fd0b80b57f0 2024-03-18T15:34:38.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.408+0000 7fd0b67fc640 1 --2- 172.21.15.5:0/1084005639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0b809f0f0 0x7fd0b80726a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd0b8074f10 tx=0x7fd0ac01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47e90d940b51f924 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:38.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.408+0000 7fd0b5ffb640 1 -- 172.21.15.5:0/1084005639 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac0b0070 con 0x7fd0b809f0f0 2024-03-18T15:34:38.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.408+0000 7fd0b5ffb640 1 -- 172.21.15.5:0/1084005639 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0ac0ab600 con 0x7fd0b809f0f0 2024-03-18T15:34:38.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.408+0000 7fd0b5ffb640 1 -- 172.21.15.5:0/1084005639 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ac0ab9d0 con 0x7fd0b809f0f0 2024-03-18T15:34:38.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.408+0000 7fd0bd47b640 1 -- 172.21.15.5:0/1084005639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0b809f0f0 msgr2=0x7fd0b80726a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:38.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.408+0000 7fd0bd47b640 1 --2- 172.21.15.5:0/1084005639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0b809f0f0 0x7fd0b80726a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd0b8074f10 tx=0x7fd0ac01c3f0 comp rx=0 tx=0).stop 2024-03-18T15:34:38.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.408+0000 7fd0bd47b640 1 -- 172.21.15.5:0/1084005639 shutdown_connections 2024-03-18T15:34:38.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.408+0000 7fd0bd47b640 1 --2- 172.21.15.5:0/1084005639 >> v2:172.21.15.80:3300/0 conn(0x7fd0b806d650 0x7fd0b8072be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:38.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.408+0000 7fd0bd47b640 1 --2- 172.21.15.5:0/1084005639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0b809f0f0 0x7fd0b80726a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:38.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.408+0000 7fd0bd47b640 1 -- 172.21.15.5:0/1084005639 >> 172.21.15.5:0/1084005639 conn(0x7fd0b80b6680 msgr2=0x7fd0b80b6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:38.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.408+0000 7fd0bd47b640 1 -- 172.21.15.5:0/1084005639 shutdown_connections 2024-03-18T15:34:38.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.408+0000 7fd0bd47b640 1 -- 172.21.15.5:0/1084005639 wait complete. 2024-03-18T15:34:38.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.408+0000 7fd0bd47b640 1 Processor -- start 2024-03-18T15:34:38.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.408+0000 7fd0bd47b640 1 -- start start 2024-03-18T15:34:38.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.412+0000 7fd0bd47b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0b806d650 0x7fd0b813a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:38.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.412+0000 7fd0bd47b640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd0b809f0f0 0x7fd0b81dd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:38.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.412+0000 7fd0bd47b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0b80b57f0 0x7fd0b81ddb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:38.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.412+0000 7fd0bd47b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b8152550 con 0x7fd0b80b57f0 2024-03-18T15:34:38.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.412+0000 7fd0bd47b640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd0b81523d0 con 0x7fd0b806d650 2024-03-18T15:34:38.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.412+0000 7fd0bd47b640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fd0b81526d0 con 0x7fd0b809f0f0 2024-03-18T15:34:38.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.412+0000 7fd0b77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0b80b57f0 0x7fd0b81ddb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:38.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.412+0000 7fd0b77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0b80b57f0 0x7fd0b81ddb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56690/0 (socket says 172.21.15.5:56690) 2024-03-18T15:34:38.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.412+0000 7fd0b77fe640 1 -- 172.21.15.5:0/1206401889 learned_addr learned my addr 172.21.15.5:0/1206401889 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:38.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.412+0000 7fd0b6ffd640 1 --2- 172.21.15.5:0/1206401889 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0b806d650 0x7fd0b813a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:38.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.412+0000 7fd0b67fc640 1 --2- 172.21.15.5:0/1206401889 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd0b809f0f0 0x7fd0b81dd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:38.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.412+0000 7fd0b77fe640 1 -- 172.21.15.5:0/1206401889 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd0b809f0f0 msgr2=0x7fd0b81dd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:38.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.412+0000 7fd0b77fe640 1 --2- 172.21.15.5:0/1206401889 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd0b809f0f0 0x7fd0b81dd5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:38.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.412+0000 7fd0b77fe640 1 -- 172.21.15.5:0/1206401889 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0b806d650 msgr2=0x7fd0b813a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:38.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.412+0000 7fd0b77fe640 1 --2- 172.21.15.5:0/1206401889 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0b806d650 0x7fd0b813a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:38.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.412+0000 7fd0b77fe640 1 -- 172.21.15.5:0/1206401889 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0b80b0f50 con 0x7fd0b80b57f0 2024-03-18T15:34:38.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.412+0000 7fd0b77fe640 1 --2- 172.21.15.5:0/1206401889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0b80b57f0 0x7fd0b81ddb00 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0007e40 tx=0x7fd0b000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:38.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.412+0000 7fd09f7fe640 1 -- 172.21.15.5:0/1206401889 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b0014620 con 0x7fd0b80b57f0 2024-03-18T15:34:38.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.412+0000 7fd09f7fe640 1 -- 172.21.15.5:0/1206401889 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0b0019070 con 0x7fd0b80b57f0 2024-03-18T15:34:38.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.412+0000 7fd0bd47b640 1 -- 172.21.15.5:0/1206401889 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0b813af80 con 0x7fd0b80b57f0 2024-03-18T15:34:38.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.412+0000 7fd09f7fe640 1 -- 172.21.15.5:0/1206401889 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b0014a20 con 0x7fd0b80b57f0 2024-03-18T15:34:38.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.412+0000 7fd0bd47b640 1 -- 172.21.15.5:0/1206401889 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0b81428b0 con 0x7fd0b80b57f0 2024-03-18T15:34:38.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.412+0000 7fd0bd47b640 1 -- 172.21.15.5:0/1206401889 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0b8139910 con 0x7fd0b80b57f0 2024-03-18T15:34:38.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.412+0000 7fd09f7fe640 1 -- 172.21.15.5:0/1206401889 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fd0b0014bc0 con 0x7fd0b80b57f0 2024-03-18T15:34:38.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.412+0000 7fd09f7fe640 1 --2- 172.21.15.5:0/1206401889 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd0a4067660 0x7fd0a4069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:38.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.412+0000 7fd09f7fe640 1 -- 172.21.15.5:0/1206401889 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5795+0+0 (secure 0 0 0) 0x7fd0b0086140 con 0x7fd0b80b57f0 2024-03-18T15:34:38.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.412+0000 7fd0b6ffd640 1 --2- 172.21.15.5:0/1206401889 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd0a4067660 0x7fd0a4069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:38.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.412+0000 7fd0b6ffd640 1 --2- 172.21.15.5:0/1206401889 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd0a4067660 0x7fd0a4069b20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8000f70 tx=0x7fd0a8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:38.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.416+0000 7fd09f7fe640 1 -- 172.21.15.5:0/1206401889 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fd0b80b0f50 con 0x7fd0b80b57f0 2024-03-18T15:34:38.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:38.580+0000 7fd0bd47b640 1 -- 172.21.15.5:0/1206401889 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd0b804f5c0 con 0x7fd0b80b57f0 2024-03-18T15:34:39.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.212+0000 7fd09f7fe640 1 -- 172.21.15.5:0/1206401889 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v32) ==== 181+0+0 (secure 0 0 0) 0x7fd0b0056df0 con 0x7fd0b80b57f0 2024-03-18T15:34:39.214 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' removed 2024-03-18T15:34:39.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.212+0000 7fd0bd47b640 1 -- 172.21.15.5:0/1206401889 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd0a4067660 msgr2=0x7fd0a4069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:39.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.212+0000 7fd0bd47b640 1 --2- 172.21.15.5:0/1206401889 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd0a4067660 0x7fd0a4069b20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd0a8000f70 tx=0x7fd0a8009040 comp rx=0 tx=0).stop 2024-03-18T15:34:39.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.212+0000 7fd0bd47b640 1 -- 172.21.15.5:0/1206401889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0b80b57f0 msgr2=0x7fd0b81ddb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:39.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.212+0000 7fd0bd47b640 1 --2- 172.21.15.5:0/1206401889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0b80b57f0 0x7fd0b81ddb00 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd0b0007e40 tx=0x7fd0b000e4b0 comp rx=0 tx=0).stop 2024-03-18T15:34:39.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.212+0000 7fd0bd47b640 1 -- 172.21.15.5:0/1206401889 shutdown_connections 2024-03-18T15:34:39.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.212+0000 7fd0bd47b640 1 --2- 172.21.15.5:0/1206401889 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd0a4067660 0x7fd0a4069b20 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:39.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.212+0000 7fd0bd47b640 1 --2- 172.21.15.5:0/1206401889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0b80b57f0 0x7fd0b81ddb00 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:39.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.212+0000 7fd0bd47b640 1 --2- 172.21.15.5:0/1206401889 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd0b809f0f0 0x7fd0b81dd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:39.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.212+0000 7fd0bd47b640 1 --2- 172.21.15.5:0/1206401889 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0b806d650 0x7fd0b813a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:39.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.212+0000 7fd0bd47b640 1 -- 172.21.15.5:0/1206401889 >> 172.21.15.5:0/1206401889 conn(0x7fd0b80b6680 msgr2=0x7fd0b80b7a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:39.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.216+0000 7fd0bd47b640 1 -- 172.21.15.5:0/1206401889 shutdown_connections 2024-03-18T15:34:39.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.216+0000 7fd0bd47b640 1 -- 172.21.15.5:0/1206401889 wait complete. 2024-03-18T15:34:39.245 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-18T15:34:39.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.376+0000 7f692c1b9640 1 Processor -- start 2024-03-18T15:34:39.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.376+0000 7f692c1b9640 1 -- start start 2024-03-18T15:34:39.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.376+0000 7f692c1b9640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f69240b4820 0x7f692413efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:39.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.376+0000 7f692c1b9640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f692406f640 0x7f692413f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:39.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.376+0000 7f692c1b9640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69240afd60 con 0x7f692406d300 2024-03-18T15:34:39.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.376+0000 7f692c1b9640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f692413fa30 con 0x7f69240b4820 2024-03-18T15:34:39.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.376+0000 7f692c1b9640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f69240b52b0 con 0x7f692406f640 2024-03-18T15:34:39.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.376+0000 7f6929f55640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f692406d300 0x7f6924145250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44380/0 (socket says 172.21.15.5:44380) 2024-03-18T15:34:39.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.376+0000 7f6929f55640 1 -- 172.21.15.5:0/4026190682 learned_addr learned my addr 172.21.15.5:0/4026190682 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:39.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.376+0000 7f692a756640 1 --2- 172.21.15.5:0/4026190682 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f692406f640 0x7f692413f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:39.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.376+0000 7f6929754640 1 --2- 172.21.15.5:0/4026190682 >> v2:172.21.15.80:3300/0 conn(0x7f69240b4820 0x7f692413efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:39.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.376+0000 7f6928f53640 1 -- 172.21.15.5:0/4026190682 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1745331685 0 0) 0x7f69240afd60 con 0x7f692406d300 2024-03-18T15:34:39.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.376+0000 7f6928f53640 1 -- 172.21.15.5:0/4026190682 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6910003530 con 0x7f692406d300 2024-03-18T15:34:39.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.376+0000 7f6928f53640 1 -- 172.21.15.5:0/4026190682 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 997313622 0 0) 0x7f6910003530 con 0x7f692406d300 2024-03-18T15:34:39.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.376+0000 7f6928f53640 1 -- 172.21.15.5:0/4026190682 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f692406f640 msgr2=0x7f692413f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:39.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.376+0000 7f6928f53640 1 --2- 172.21.15.5:0/4026190682 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f692406f640 0x7f692413f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:39.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.376+0000 7f6928f53640 1 -- 172.21.15.5:0/4026190682 >> v2:172.21.15.80:3300/0 conn(0x7f69240b4820 msgr2=0x7f692413efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:39.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.376+0000 7f6928f53640 1 --2- 172.21.15.5:0/4026190682 >> v2:172.21.15.80:3300/0 conn(0x7f69240b4820 0x7f692413efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:39.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.376+0000 7f6928f53640 1 -- 172.21.15.5:0/4026190682 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f692406d6e0 con 0x7f692406d300 2024-03-18T15:34:39.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.376+0000 7f6928f53640 1 -- 172.21.15.5:0/4026190682 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3837549003 0 0) 0x7f6920001680 con 0x7f692406d300 2024-03-18T15:34:39.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.376+0000 7f6928f53640 1 -- 172.21.15.5:0/4026190682 >> v1:172.21.15.5:6789/0 conn(0x7f692406d300 legacy=0x7f6924145250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:39.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.376+0000 7f6928f53640 1 --2- 172.21.15.5:0/4026190682 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f692406f640 0x7f6910003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:39.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.376+0000 7f6928f53640 1 --2- 172.21.15.5:0/4026190682 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6910004020 0x7f6910004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:39.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.376+0000 7f6928f53640 1 --2- 172.21.15.5:0/4026190682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6910004d50 0x7f69100051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:39.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.376+0000 7f6928f53640 1 -- 172.21.15.5:0/4026190682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f69240b52b0 con 0x7f6910004d50 2024-03-18T15:34:39.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.376+0000 7f6928f53640 1 -- 172.21.15.5:0/4026190682 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f691000bb10 con 0x7f6910004020 2024-03-18T15:34:39.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.376+0000 7f6928f53640 1 -- 172.21.15.5:0/4026190682 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f691000bd10 con 0x7f692406f640 2024-03-18T15:34:39.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.376+0000 7f692a756640 1 --2- 172.21.15.5:0/4026190682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6910004d50 0x7f69100051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:39.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.376+0000 7f6929754640 1 --2- 172.21.15.5:0/4026190682 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6910004020 0x7f6910004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:39.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.376+0000 7f6929f55640 1 --2- 172.21.15.5:0/4026190682 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f692406f640 0x7f6910003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:39.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.376+0000 7f692a756640 1 -- 172.21.15.5:0/4026190682 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f692406f640 msgr2=0x7f6910003aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:39.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.376+0000 7f692a756640 1 --2- 172.21.15.5:0/4026190682 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f692406f640 0x7f6910003aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:39.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.376+0000 7f692a756640 1 -- 172.21.15.5:0/4026190682 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6910004020 msgr2=0x7f6910004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:39.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.376+0000 7f692a756640 1 --2- 172.21.15.5:0/4026190682 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6910004020 0x7f6910004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:39.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.376+0000 7f692a756640 1 -- 172.21.15.5:0/4026190682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6910003530 con 0x7f6910004d50 2024-03-18T15:34:39.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.380+0000 7f692a756640 1 --2- 172.21.15.5:0/4026190682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6910004d50 0x7f69100051d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f691c009f90 tx=0x7f691c01cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c582f60d1bdba5c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:39.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.380+0000 7f6928f53640 1 -- 172.21.15.5:0/4026190682 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691c0b2070 con 0x7f6910004d50 2024-03-18T15:34:39.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.380+0000 7f6928f53640 1 -- 172.21.15.5:0/4026190682 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f691c0a5b10 con 0x7f6910004d50 2024-03-18T15:34:39.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.380+0000 7f6928f53640 1 -- 172.21.15.5:0/4026190682 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691c009c10 con 0x7f6910004d50 2024-03-18T15:34:39.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.380+0000 7f692c1b9640 1 -- 172.21.15.5:0/4026190682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6910004d50 msgr2=0x7f69100051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:39.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.380+0000 7f692c1b9640 1 --2- 172.21.15.5:0/4026190682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6910004d50 0x7f69100051d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f691c009f90 tx=0x7f691c01cc10 comp rx=0 tx=0).stop 2024-03-18T15:34:39.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.380+0000 7f692a756640 1 -- 172.21.15.5:0/4026190682 reap_dead start 2024-03-18T15:34:39.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.380+0000 7f692c1b9640 1 -- 172.21.15.5:0/4026190682 shutdown_connections 2024-03-18T15:34:39.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.380+0000 7f692c1b9640 1 -- 172.21.15.5:0/4026190682 >> 172.21.15.5:0/4026190682 conn(0x7f69240b5670 msgr2=0x7f69240b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:39.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.380+0000 7f692c1b9640 1 -- 172.21.15.5:0/4026190682 shutdown_connections 2024-03-18T15:34:39.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.380+0000 7f692c1b9640 1 -- 172.21.15.5:0/4026190682 wait complete. 2024-03-18T15:34:39.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.380+0000 7f692c1b9640 1 Processor -- start 2024-03-18T15:34:39.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.384+0000 7f692c1b9640 1 -- start start 2024-03-18T15:34:39.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.384+0000 7f692c1b9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6924144470 0x7f6924140140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:39.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.384+0000 7f692c1b9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6924139da0 0x7f692413a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:39.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.384+0000 7f692c1b9640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f69240b0070 0x7f69240b0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:39.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.384+0000 7f692c1b9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f69240b2a70 con 0x7f6924144470 2024-03-18T15:34:39.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.384+0000 7f692c1b9640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f69240b28f0 con 0x7f6924139da0 2024-03-18T15:34:39.387 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.384+0000 7f692c1b9640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f6924150340 con 0x7f69240b0070 2024-03-18T15:34:39.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.384+0000 7f6929f55640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6924144470 0x7f6924140140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:39.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.384+0000 7f6929f55640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6924144470 0x7f6924140140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56712/0 (socket says 172.21.15.5:56712) 2024-03-18T15:34:39.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.384+0000 7f6929f55640 1 -- 172.21.15.5:0/1006803208 learned_addr learned my addr 172.21.15.5:0/1006803208 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:39.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.384+0000 7f6929754640 1 --2- 172.21.15.5:0/1006803208 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6924139da0 0x7f692413a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:39.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.384+0000 7f692a756640 1 --2- 172.21.15.5:0/1006803208 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f69240b0070 0x7f69240b0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:39.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.384+0000 7f6929f55640 1 -- 172.21.15.5:0/1006803208 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f69240b0070 msgr2=0x7f69240b0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:39.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.384+0000 7f6929f55640 1 --2- 172.21.15.5:0/1006803208 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f69240b0070 0x7f69240b0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:39.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.384+0000 7f6929f55640 1 -- 172.21.15.5:0/1006803208 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6924139da0 msgr2=0x7f692413a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:39.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.384+0000 7f6929f55640 1 --2- 172.21.15.5:0/1006803208 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6924139da0 0x7f692413a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:39.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.384+0000 7f6929f55640 1 -- 172.21.15.5:0/1006803208 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6924144850 con 0x7f6924144470 2024-03-18T15:34:39.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.384+0000 7f6929754640 1 --2- 172.21.15.5:0/1006803208 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6924139da0 0x7f692413a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:34:39.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.384+0000 7f692a756640 1 --2- 172.21.15.5:0/1006803208 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f69240b0070 0x7f69240b0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:34:39.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.384+0000 7f6929f55640 1 --2- 172.21.15.5:0/1006803208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6924144470 0x7f6924140140 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6920004fc0 tx=0x7f69200074f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:39.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.384+0000 7f691a7fc640 1 -- 172.21.15.5:0/1006803208 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6920007740 con 0x7f6924144470 2024-03-18T15:34:39.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.384+0000 7f691a7fc640 1 -- 172.21.15.5:0/1006803208 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6920018070 con 0x7f6924144470 2024-03-18T15:34:39.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.384+0000 7f692c1b9640 1 -- 172.21.15.5:0/1006803208 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69240b18b0 con 0x7f6924144470 2024-03-18T15:34:39.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.384+0000 7f691a7fc640 1 -- 172.21.15.5:0/1006803208 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692000fbc0 con 0x7f6924144470 2024-03-18T15:34:39.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.384+0000 7f692c1b9640 1 -- 172.21.15.5:0/1006803208 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6924142100 con 0x7f6924144470 2024-03-18T15:34:39.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.384+0000 7f692c1b9640 1 -- 172.21.15.5:0/1006803208 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f692404f5c0 con 0x7f6924144470 2024-03-18T15:34:39.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.384+0000 7f691a7fc640 1 -- 172.21.15.5:0/1006803208 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f692001d460 con 0x7f6924144470 2024-03-18T15:34:39.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.384+0000 7f691a7fc640 1 --2- 172.21.15.5:0/1006803208 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6908067690 0x7f6908069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:39.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.384+0000 7f691a7fc640 1 -- 172.21.15.5:0/1006803208 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5357+0+0 (secure 0 0 0) 0x7f6920085830 con 0x7f6924144470 2024-03-18T15:34:39.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.384+0000 7f6929754640 1 --2- 172.21.15.5:0/1006803208 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6908067690 0x7f6908069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:39.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.388+0000 7f6929754640 1 --2- 172.21.15.5:0/1006803208 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6908067690 0x7f6908069b50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6924073020 tx=0x7f6914008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:39.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.388+0000 7f691a7fc640 1 -- 172.21.15.5:0/1006803208 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f6920056610 con 0x7f6924144470 2024-03-18T15:34:39.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:39.552+0000 7f692c1b9640 1 -- 172.21.15.5:0/1006803208 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6924071e90 con 0x7f6924144470 2024-03-18T15:34:40.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.212+0000 7f691a7fc640 1 -- 172.21.15.5:0/1006803208 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v33) ==== 169+0+0 (secure 0 0 0) 0x7f6920055f10 con 0x7f6924144470 2024-03-18T15:34:40.216 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' removed 2024-03-18T15:34:40.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.216+0000 7f692c1b9640 1 -- 172.21.15.5:0/1006803208 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6908067690 msgr2=0x7f6908069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:40.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.216+0000 7f692c1b9640 1 --2- 172.21.15.5:0/1006803208 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6908067690 0x7f6908069b50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6924073020 tx=0x7f6914008040 comp rx=0 tx=0).stop 2024-03-18T15:34:40.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.216+0000 7f692c1b9640 1 -- 172.21.15.5:0/1006803208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6924144470 msgr2=0x7f6924140140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:40.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.216+0000 7f692c1b9640 1 --2- 172.21.15.5:0/1006803208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6924144470 0x7f6924140140 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6920004fc0 tx=0x7f69200074f0 comp rx=0 tx=0).stop 2024-03-18T15:34:40.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.220+0000 7f692c1b9640 1 -- 172.21.15.5:0/1006803208 shutdown_connections 2024-03-18T15:34:40.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.220+0000 7f692c1b9640 1 --2- 172.21.15.5:0/1006803208 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f6908067690 0x7f6908069b50 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:40.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.220+0000 7f692c1b9640 1 --2- 172.21.15.5:0/1006803208 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f69240b0070 0x7f69240b0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:40.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.220+0000 7f692c1b9640 1 --2- 172.21.15.5:0/1006803208 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6924139da0 0x7f692413a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:40.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.220+0000 7f692c1b9640 1 --2- 172.21.15.5:0/1006803208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6924144470 0x7f6924140140 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:40.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.220+0000 7f692c1b9640 1 -- 172.21.15.5:0/1006803208 >> 172.21.15.5:0/1006803208 conn(0x7f69240b5670 msgr2=0x7f692406c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:40.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.220+0000 7f692c1b9640 1 -- 172.21.15.5:0/1006803208 shutdown_connections 2024-03-18T15:34:40.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.220+0000 7f692c1b9640 1 -- 172.21.15.5:0/1006803208 wait complete. 2024-03-18T15:34:40.244 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-18T15:34:40.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.360+0000 7f9224c24640 1 Processor -- start 2024-03-18T15:34:40.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.360+0000 7f9224c24640 1 -- start start 2024-03-18T15:34:40.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.364+0000 7f9224c24640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9220072b40 0x7f922006d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:40.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.364+0000 7f9224c24640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92200afb70 con 0x7f92200af790 2024-03-18T15:34:40.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.364+0000 7f9224c24640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92200b4a20 con 0x7f92200b26a0 2024-03-18T15:34:40.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.364+0000 7f9224c24640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f922006d5b0 con 0x7f9220072b40 2024-03-18T15:34:40.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.364+0000 7f921dd9b640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f9220072b40 0x7f922006d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:40.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.364+0000 7f921e59c640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f92200af790 0x7f9220147780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:44390/0 (socket says 172.21.15.5:44390) 2024-03-18T15:34:40.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.364+0000 7f921e59c640 1 -- 172.21.15.5:0/3392542722 learned_addr learned my addr 172.21.15.5:0/3392542722 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:40.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.364+0000 7f921dd9b640 1 -- 172.21.15.5:0/3392542722 >> v1:172.21.15.80:6789/0 conn(0x7f92200b26a0 legacy=0x7f92200b2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:40.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.364+0000 7f921dd9b640 1 -- 172.21.15.5:0/3392542722 >> v1:172.21.15.5:6789/0 conn(0x7f92200af790 legacy=0x7f9220147780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:40.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.364+0000 7f921dd9b640 1 -- 172.21.15.5:0/3392542722 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92200b4d00 con 0x7f9220072b40 2024-03-18T15:34:40.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.364+0000 7f921dd9b640 1 --2- 172.21.15.5:0/3392542722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9220072b40 0x7f922006d070 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f92140097c0 tx=0x7f921401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8802b9ffb5f9d969 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:40.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.364+0000 7f921d59a640 1 -- 172.21.15.5:0/3392542722 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92140b0070 con 0x7f9220072b40 2024-03-18T15:34:40.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.364+0000 7f921d59a640 1 -- 172.21.15.5:0/3392542722 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92140ab600 con 0x7f9220072b40 2024-03-18T15:34:40.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.364+0000 7f921d59a640 1 -- 172.21.15.5:0/3392542722 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92140ab9d0 con 0x7f9220072b40 2024-03-18T15:34:40.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.368+0000 7f9224c24640 1 -- 172.21.15.5:0/3392542722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9220072b40 msgr2=0x7f922006d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:40.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.368+0000 7f9224c24640 1 --2- 172.21.15.5:0/3392542722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9220072b40 0x7f922006d070 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f92140097c0 tx=0x7f921401c3d0 comp rx=0 tx=0).stop 2024-03-18T15:34:40.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.368+0000 7f9224c24640 1 -- 172.21.15.5:0/3392542722 shutdown_connections 2024-03-18T15:34:40.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.368+0000 7f9224c24640 1 --2- 172.21.15.5:0/3392542722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9220072b40 0x7f922006d070 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:40.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.368+0000 7f9224c24640 1 -- 172.21.15.5:0/3392542722 >> 172.21.15.5:0/3392542722 conn(0x7f92200b5670 msgr2=0x7f92200b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:40.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.368+0000 7f9224c24640 1 -- 172.21.15.5:0/3392542722 shutdown_connections 2024-03-18T15:34:40.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.368+0000 7f9224c24640 1 -- 172.21.15.5:0/3392542722 wait complete. 2024-03-18T15:34:40.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.368+0000 7f9224c24640 1 Processor -- start 2024-03-18T15:34:40.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.368+0000 7f9224c24640 1 -- start start 2024-03-18T15:34:40.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.368+0000 7f9224c24640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9220072b40 0x7f9220138f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:40.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.368+0000 7f9224c24640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92200af790 0x7f92201e11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:40.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.368+0000 7f9224c24640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f92200b26a0 0x7f9220135bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:40.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.368+0000 7f9224c24640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9220152ac0 con 0x7f9220072b40 2024-03-18T15:34:40.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.368+0000 7f9224c24640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9220152940 con 0x7f92200af790 2024-03-18T15:34:40.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.368+0000 7f9224c24640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f9220152c40 con 0x7f92200b26a0 2024-03-18T15:34:40.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.368+0000 7f921e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9220072b40 0x7f9220138f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:40.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.368+0000 7f921e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9220072b40 0x7f9220138f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56732/0 (socket says 172.21.15.5:56732) 2024-03-18T15:34:40.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.368+0000 7f921e59c640 1 -- 172.21.15.5:0/489600051 learned_addr learned my addr 172.21.15.5:0/489600051 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:40.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.368+0000 7f921dd9b640 1 --2- 172.21.15.5:0/489600051 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92200af790 0x7f92201e11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:40.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.368+0000 7f921e59c640 1 -- 172.21.15.5:0/489600051 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f92200b26a0 msgr2=0x7f9220135bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:40.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.368+0000 7f921e59c640 1 --2- 172.21.15.5:0/489600051 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f92200b26a0 0x7f9220135bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:40.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.368+0000 7f921e59c640 1 -- 172.21.15.5:0/489600051 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92200af790 msgr2=0x7f92201e11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:40.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.368+0000 7f921e59c640 1 --2- 172.21.15.5:0/489600051 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92200af790 0x7f92201e11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:40.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.368+0000 7f921e59c640 1 -- 172.21.15.5:0/489600051 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f922014f7c0 con 0x7f9220072b40 2024-03-18T15:34:40.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.368+0000 7f921dd9b640 1 --2- 172.21.15.5:0/489600051 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92200af790 0x7f92201e11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:40.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.368+0000 7f921e59c640 1 --2- 172.21.15.5:0/489600051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9220072b40 0x7f9220138f10 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f921000c420 tx=0x7f921000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:40.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.368+0000 7f920effd640 1 -- 172.21.15.5:0/489600051 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9210017070 con 0x7f9220072b40 2024-03-18T15:34:40.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.368+0000 7f920effd640 1 -- 172.21.15.5:0/489600051 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9210005cf0 con 0x7f9220072b40 2024-03-18T15:34:40.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.368+0000 7f9224c24640 1 -- 172.21.15.5:0/489600051 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f922013fd80 con 0x7f9220072b40 2024-03-18T15:34:40.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.368+0000 7f920effd640 1 -- 172.21.15.5:0/489600051 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92100072f0 con 0x7f9220072b40 2024-03-18T15:34:40.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.368+0000 7f9224c24640 1 -- 172.21.15.5:0/489600051 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9220139b60 con 0x7f9220072b40 2024-03-18T15:34:40.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.368+0000 7f9224c24640 1 -- 172.21.15.5:0/489600051 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f922004f550 con 0x7f9220072b40 2024-03-18T15:34:40.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.372+0000 7f920effd640 1 -- 172.21.15.5:0/489600051 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f921001a070 con 0x7f9220072b40 2024-03-18T15:34:40.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.372+0000 7f920effd640 1 --2- 172.21.15.5:0/489600051 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f9200067660 0x7f9200069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:40.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.372+0000 7f920effd640 1 -- 172.21.15.5:0/489600051 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7f92100849c0 con 0x7f9220072b40 2024-03-18T15:34:40.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.372+0000 7f921dd9b640 1 --2- 172.21.15.5:0/489600051 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f9200067660 0x7f9200069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:40.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.372+0000 7f921dd9b640 1 --2- 172.21.15.5:0/489600051 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f9200067660 0x7f9200069b20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f92140ad000 tx=0x7f9214008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:40.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.376+0000 7f920effd640 1 -- 172.21.15.5:0/489600051 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f9210086e20 con 0x7f9220072b40 2024-03-18T15:34:40.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.536+0000 7f9224c24640 1 -- 172.21.15.5:0/489600051 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f92200b22e0 con 0x7f9220072b40 2024-03-18T15:34:40.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.540+0000 7f920effd640 1 -- 172.21.15.5:0/489600051 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f9210055940 con 0x7f9220072b40 2024-03-18T15:34:40.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.544+0000 7f9224c24640 1 -- 172.21.15.5:0/489600051 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f9200067660 msgr2=0x7f9200069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:40.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.544+0000 7f9224c24640 1 --2- 172.21.15.5:0/489600051 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f9200067660 0x7f9200069b20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f92140ad000 tx=0x7f9214008040 comp rx=0 tx=0).stop 2024-03-18T15:34:40.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.544+0000 7f9224c24640 1 -- 172.21.15.5:0/489600051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9220072b40 msgr2=0x7f9220138f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:40.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.544+0000 7f9224c24640 1 --2- 172.21.15.5:0/489600051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9220072b40 0x7f9220138f10 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f921000c420 tx=0x7f921000c8e0 comp rx=0 tx=0).stop 2024-03-18T15:34:40.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.544+0000 7f9224c24640 1 -- 172.21.15.5:0/489600051 shutdown_connections 2024-03-18T15:34:40.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.544+0000 7f9224c24640 1 --2- 172.21.15.5:0/489600051 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f9200067660 0x7f9200069b20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:40.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.544+0000 7f9224c24640 1 --2- 172.21.15.5:0/489600051 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f92200b26a0 0x7f9220135bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:40.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.544+0000 7f9224c24640 1 --2- 172.21.15.5:0/489600051 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92200af790 0x7f92201e11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:40.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.544+0000 7f9224c24640 1 --2- 172.21.15.5:0/489600051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9220072b40 0x7f9220138f10 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:40.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.544+0000 7f9224c24640 1 -- 172.21.15.5:0/489600051 >> 172.21.15.5:0/489600051 conn(0x7f92200b5670 msgr2=0x7f92200b8310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:40.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.544+0000 7f9224c24640 1 -- 172.21.15.5:0/489600051 shutdown_connections 2024-03-18T15:34:40.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.544+0000 7f9224c24640 1 -- 172.21.15.5:0/489600051 wait complete. 2024-03-18T15:34:40.565 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_cephfs_shell.TestCD.test_cd_with_args' 2024-03-18T15:34:40.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.684+0000 7f1be5a49640 1 Processor -- start 2024-03-18T15:34:40.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.684+0000 7f1be5a49640 1 -- start start 2024-03-18T15:34:40.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.684+0000 7f1be5a49640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1be00aec00 0x7f1be00aefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:40.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.684+0000 7f1be5a49640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1be0143bd0 0x7f1be0143ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:40.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.684+0000 7f1be5a49640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1be00add60 con 0x7f1be0147060 2024-03-18T15:34:40.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.684+0000 7f1be5a49640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f1be00ae090 con 0x7f1be00aec00 2024-03-18T15:34:40.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.684+0000 7f1be5a49640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f1be00b0350 con 0x7f1be0143bd0 2024-03-18T15:34:40.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.684+0000 7f1bdeffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1be00aec00 0x7f1be00aefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:40.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.684+0000 7f1bdeffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f1be00aec00 0x7f1be00aefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33954/0 (socket says 172.21.15.5:33954) 2024-03-18T15:34:40.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.684+0000 7f1bdeffd640 1 -- 172.21.15.5:0/1290190253 learned_addr learned my addr 172.21.15.5:0/1290190253 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:40.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.688+0000 7f1bde7fc640 1 --2- 172.21.15.5:0/1290190253 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1be0143bd0 0x7f1be0143ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:40.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.688+0000 7f1bdeffd640 1 -- 172.21.15.5:0/1290190253 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1be0143bd0 msgr2=0x7f1be0143ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:40.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.688+0000 7f1bdeffd640 1 --2- 172.21.15.5:0/1290190253 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1be0143bd0 0x7f1be0143ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:40.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.688+0000 7f1bdeffd640 1 -- 172.21.15.5:0/1290190253 >> v1:172.21.15.80:6789/0 conn(0x7f1be0147060 legacy=0x7f1be01474d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:40.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.688+0000 7f1bdeffd640 1 -- 172.21.15.5:0/1290190253 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1be00b0630 con 0x7f1be00aec00 2024-03-18T15:34:40.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.688+0000 7f1bdeffd640 1 --2- 172.21.15.5:0/1290190253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1be00aec00 0x7f1be00aefe0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1bd000bb10 tx=0x7f1bd001ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e258c198d8bcb9e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:40.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.688+0000 7f1bddffb640 1 -- 172.21.15.5:0/1290190253 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd00b4070 con 0x7f1be00aec00 2024-03-18T15:34:40.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.688+0000 7f1bddffb640 1 -- 172.21.15.5:0/1290190253 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bd00a7b90 con 0x7f1be00aec00 2024-03-18T15:34:40.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.688+0000 7f1bddffb640 1 -- 172.21.15.5:0/1290190253 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd00af440 con 0x7f1be00aec00 2024-03-18T15:34:40.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.688+0000 7f1be5a49640 1 -- 172.21.15.5:0/1290190253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1be00aec00 msgr2=0x7f1be00aefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:40.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.688+0000 7f1be5a49640 1 --2- 172.21.15.5:0/1290190253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1be00aec00 0x7f1be00aefe0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1bd000bb10 tx=0x7f1bd001ec30 comp rx=0 tx=0).stop 2024-03-18T15:34:40.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.688+0000 7f1be5a49640 1 -- 172.21.15.5:0/1290190253 shutdown_connections 2024-03-18T15:34:40.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.688+0000 7f1be5a49640 1 --2- 172.21.15.5:0/1290190253 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1be0143bd0 0x7f1be0143ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:40.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.688+0000 7f1be5a49640 1 --2- 172.21.15.5:0/1290190253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1be00aec00 0x7f1be00aefe0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:40.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.688+0000 7f1be5a49640 1 -- 172.21.15.5:0/1290190253 >> 172.21.15.5:0/1290190253 conn(0x7f1be00b5670 msgr2=0x7f1be00b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:40.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.688+0000 7f1be5a49640 1 -- 172.21.15.5:0/1290190253 shutdown_connections 2024-03-18T15:34:40.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.688+0000 7f1be5a49640 1 -- 172.21.15.5:0/1290190253 wait complete. 2024-03-18T15:34:40.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.692+0000 7f1be5a49640 1 Processor -- start 2024-03-18T15:34:40.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.692+0000 7f1be5a49640 1 -- start start 2024-03-18T15:34:40.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.692+0000 7f1be5a49640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1be00aec00 0x7f1be0136760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:40.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.692+0000 7f1be5a49640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1be0143bd0 0x7f1be00b1620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:40.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.692+0000 7f1be5a49640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1be0147060 0x7f1be00b1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:40.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.692+0000 7f1be5a49640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1be01507a0 con 0x7f1be0147060 2024-03-18T15:34:40.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.692+0000 7f1be5a49640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1be0150620 con 0x7f1be0143bd0 2024-03-18T15:34:40.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.692+0000 7f1be5a49640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f1be0150920 con 0x7f1be00aec00 2024-03-18T15:34:40.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.692+0000 7f1bdf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1be0147060 0x7f1be00b1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:40.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.692+0000 7f1bdf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1be0147060 0x7f1be00b1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33956/0 (socket says 172.21.15.5:33956) 2024-03-18T15:34:40.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.692+0000 7f1bdf7fe640 1 -- 172.21.15.5:0/2791182841 learned_addr learned my addr 172.21.15.5:0/2791182841 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:40.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.692+0000 7f1bdeffd640 1 --2- 172.21.15.5:0/2791182841 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1be00aec00 0x7f1be0136760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:40.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.692+0000 7f1bde7fc640 1 --2- 172.21.15.5:0/2791182841 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1be0143bd0 0x7f1be00b1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:40.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.692+0000 7f1bdf7fe640 1 -- 172.21.15.5:0/2791182841 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1be00aec00 msgr2=0x7f1be0136760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:40.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.692+0000 7f1bdf7fe640 1 --2- 172.21.15.5:0/2791182841 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1be00aec00 0x7f1be0136760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:40.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.692+0000 7f1bdf7fe640 1 -- 172.21.15.5:0/2791182841 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1be0143bd0 msgr2=0x7f1be00b1620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:40.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.692+0000 7f1bdf7fe640 1 --2- 172.21.15.5:0/2791182841 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1be0143bd0 0x7f1be00b1620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:40.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.692+0000 7f1bdf7fe640 1 -- 172.21.15.5:0/2791182841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1be00b0ab0 con 0x7f1be0147060 2024-03-18T15:34:40.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.692+0000 7f1bdf7fe640 1 --2- 172.21.15.5:0/2791182841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1be0147060 0x7f1be00b1b60 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1bd800c770 tx=0x7f1bd800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:40.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.692+0000 7f1bc7fff640 1 -- 172.21.15.5:0/2791182841 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd8003b80 con 0x7f1be0147060 2024-03-18T15:34:40.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.692+0000 7f1bc7fff640 1 -- 172.21.15.5:0/2791182841 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bd8003d20 con 0x7f1be0147060 2024-03-18T15:34:40.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.692+0000 7f1be5a49640 1 -- 172.21.15.5:0/2791182841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1be00b0d90 con 0x7f1be0147060 2024-03-18T15:34:40.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.692+0000 7f1bc7fff640 1 -- 172.21.15.5:0/2791182841 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd800aa30 con 0x7f1be0147060 2024-03-18T15:34:40.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.692+0000 7f1be5a49640 1 -- 172.21.15.5:0/2791182841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1be01467e0 con 0x7f1be0147060 2024-03-18T15:34:40.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.692+0000 7f1be5a49640 1 -- 172.21.15.5:0/2791182841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1be00b1300 con 0x7f1be0147060 2024-03-18T15:34:40.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.692+0000 7f1bc7fff640 1 -- 172.21.15.5:0/2791182841 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f1bd800abd0 con 0x7f1be0147060 2024-03-18T15:34:40.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.696+0000 7f1bc7fff640 1 --2- 172.21.15.5:0/2791182841 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f1bcc067660 0x7f1bcc069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:40.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.696+0000 7f1bc7fff640 1 -- 172.21.15.5:0/2791182841 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7f1bd80854a0 con 0x7f1be0147060 2024-03-18T15:34:40.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.696+0000 7f1bdeffd640 1 --2- 172.21.15.5:0/2791182841 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f1bcc067660 0x7f1bcc069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:40.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.696+0000 7f1bdeffd640 1 --2- 172.21.15.5:0/2791182841 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f1bcc067660 0x7f1bcc069b20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1bd00b1000 tx=0x7f1bd0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:40.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.700+0000 7f1bc7fff640 1 -- 172.21.15.5:0/2791182841 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f1be00b1300 con 0x7f1be0147060 2024-03-18T15:34:40.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:40.920+0000 7f1be5a49640 1 -- 172.21.15.5:0/2791182841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_cephfs_shell.TestCD.test_cd_with_args"]} v 0) -- 0x7f1be004f550 con 0x7f1be0147060 2024-03-18T15:34:41.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.216+0000 7f1bc7fff640 1 -- 172.21.15.5:0/2791182841 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_cephfs_shell.TestCD.test_cd_with_args"]}]=0 v79) ==== 134+0+0 (secure 0 0 0) 0x7f1bd8056420 con 0x7f1be0147060 2024-03-18T15:34:41.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.220+0000 7f1be5a49640 1 -- 172.21.15.5:0/2791182841 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f1bcc067660 msgr2=0x7f1bcc069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:41.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.220+0000 7f1be5a49640 1 --2- 172.21.15.5:0/2791182841 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f1bcc067660 0x7f1bcc069b20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1bd00b1000 tx=0x7f1bd0008040 comp rx=0 tx=0).stop 2024-03-18T15:34:41.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.220+0000 7f1be5a49640 1 -- 172.21.15.5:0/2791182841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1be0147060 msgr2=0x7f1be00b1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:41.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.220+0000 7f1be5a49640 1 --2- 172.21.15.5:0/2791182841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1be0147060 0x7f1be00b1b60 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1bd800c770 tx=0x7f1bd800cc30 comp rx=0 tx=0).stop 2024-03-18T15:34:41.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.220+0000 7f1be5a49640 1 -- 172.21.15.5:0/2791182841 shutdown_connections 2024-03-18T15:34:41.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.220+0000 7f1be5a49640 1 --2- 172.21.15.5:0/2791182841 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f1bcc067660 0x7f1bcc069b20 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:41.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.220+0000 7f1be5a49640 1 --2- 172.21.15.5:0/2791182841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1be0147060 0x7f1be00b1b60 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:41.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.220+0000 7f1be5a49640 1 --2- 172.21.15.5:0/2791182841 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1be0143bd0 0x7f1be00b1620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:41.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.220+0000 7f1be5a49640 1 --2- 172.21.15.5:0/2791182841 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1be00aec00 0x7f1be0136760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:41.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.220+0000 7f1be5a49640 1 -- 172.21.15.5:0/2791182841 >> 172.21.15.5:0/2791182841 conn(0x7f1be00b5670 msgr2=0x7f1be013d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:41.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.220+0000 7f1be5a49640 1 -- 172.21.15.5:0/2791182841 shutdown_connections 2024-03-18T15:34:41.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.220+0000 7f1be5a49640 1 -- 172.21.15.5:0/2791182841 wait complete. 2024-03-18T15:34:41.242 INFO:tasks.cephfs_test_runner:Test that when cd is issued with an argument, CWD is changed ... ERROR 2024-03-18T15:34:41.243 INFO:tasks.cephfs_test_runner: 2024-03-18T15:34:41.243 INFO:tasks.cephfs_test_runner:====================================================================== 2024-03-18T15:34:41.243 INFO:tasks.cephfs_test_runner:ERROR: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD) 2024-03-18T15:34:41.243 INFO:tasks.cephfs_test_runner:Test that when cd is issued with an argument, CWD is changed 2024-03-18T15:34:41.243 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-18T15:34:41.243 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-03-18T15:34:41.243 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_4e19f704afd9957ea657ab858f9ecf76401fef6f/qa/tasks/cephfs/test_cephfs_shell.py", line 602, in test_cd_with_args 2024-03-18T15:34:41.243 INFO:tasks.cephfs_test_runner: output = self.get_cephfs_shell_script_output(script) 2024-03-18T15:34:41.243 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_4e19f704afd9957ea657ab858f9ecf76401fef6f/qa/tasks/cephfs/test_cephfs_shell.py", line 161, in get_cephfs_shell_script_output 2024-03-18T15:34:41.243 INFO:tasks.cephfs_test_runner: return ensure_str(self.run_cephfs_shell_script( 2024-03-18T15:34:41.243 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_4e19f704afd9957ea657ab858f9ecf76401fef6f/qa/tasks/cephfs/test_cephfs_shell.py", line 153, in run_cephfs_shell_script 2024-03-18T15:34:41.243 INFO:tasks.cephfs_test_runner: return mount_x.client_remote.run(args=args, stdout=stdout, 2024-03-18T15:34:41.243 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run 2024-03-18T15:34:41.243 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-03-18T15:34:41.244 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 455, in run 2024-03-18T15:34:41.244 INFO:tasks.cephfs_test_runner: r.wait() 2024-03-18T15:34:41.244 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait 2024-03-18T15:34:41.244 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-03-18T15:34:41.244 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-03-18T15:34:41.244 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-03-18T15:34:41.244 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi005 with status 127: 'cephfs-shell -b /tmp/test-cephfs12wtjnq0' 2024-03-18T15:34:41.244 INFO:tasks.cephfs_test_runner: 2024-03-18T15:34:41.244 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-18T15:34:41.244 INFO:tasks.cephfs_test_runner:Ran 1 test in 38.384s 2024-03-18T15:34:41.244 INFO:tasks.cephfs_test_runner: 2024-03-18T15:34:41.244 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-03-18T15:34:41.244 INFO:tasks.cephfs_test_runner: 2024-03-18T15:34:41.244 INFO:tasks.cephfs_test_runner:====================================================================== 2024-03-18T15:34:41.244 INFO:tasks.cephfs_test_runner:ERROR: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD) 2024-03-18T15:34:41.244 INFO:tasks.cephfs_test_runner:Test that when cd is issued with an argument, CWD is changed 2024-03-18T15:34:41.245 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-18T15:34:41.245 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-03-18T15:34:41.245 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_4e19f704afd9957ea657ab858f9ecf76401fef6f/qa/tasks/cephfs/test_cephfs_shell.py", line 602, in test_cd_with_args 2024-03-18T15:34:41.245 INFO:tasks.cephfs_test_runner: output = self.get_cephfs_shell_script_output(script) 2024-03-18T15:34:41.245 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_4e19f704afd9957ea657ab858f9ecf76401fef6f/qa/tasks/cephfs/test_cephfs_shell.py", line 161, in get_cephfs_shell_script_output 2024-03-18T15:34:41.245 INFO:tasks.cephfs_test_runner: return ensure_str(self.run_cephfs_shell_script( 2024-03-18T15:34:41.245 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_4e19f704afd9957ea657ab858f9ecf76401fef6f/qa/tasks/cephfs/test_cephfs_shell.py", line 153, in run_cephfs_shell_script 2024-03-18T15:34:41.245 INFO:tasks.cephfs_test_runner: return mount_x.client_remote.run(args=args, stdout=stdout, 2024-03-18T15:34:41.245 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run 2024-03-18T15:34:41.245 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-03-18T15:34:41.245 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 455, in run 2024-03-18T15:34:41.245 INFO:tasks.cephfs_test_runner: r.wait() 2024-03-18T15:34:41.245 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait 2024-03-18T15:34:41.245 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-03-18T15:34:41.245 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-03-18T15:34:41.245 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-03-18T15:34:41.245 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi005 with status 127: 'cephfs-shell -b /tmp/test-cephfs12wtjnq0' 2024-03-18T15:34:41.245 INFO:tasks.cephfs_test_runner: 2024-03-18T15:34:41.246 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_batrick_ceph_4e19f704afd9957ea657ab858f9ecf76401fef6f/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD) 2024-03-18T15:34:41.485 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=08087e8ed7a5413ab8e57fdadb076eb4 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_batrick_ceph_4e19f704afd9957ea657ab858f9ecf76401fef6f/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD) 2024-03-18T15:34:41.486 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-03-18T15:34:41.496 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-03-18T15:34:41.504 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-03-18T15:34:41.504 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-03-18T15:34:41.504 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-18T15:34:41.519 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339616k,nr_inodes=4084904,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275316k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17471 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275312k,nr_inodes=818828,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275316k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-18T15:34:41.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:34:41.519 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-03-18T15:34:41.567 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-18T15:34:41.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:34:41.567 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-18T15:34:41.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-18T15:34:41.624 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link delete ceph-brx 2024-03-18T15:34:41.799 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-03-18T15:34:41.808 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-18T15:34:41.808 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-18T15:34:41.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.936+0000 7f0fd2082640 1 Processor -- start 2024-03-18T15:34:41.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.936+0000 7f0fd2082640 1 -- start start 2024-03-18T15:34:41.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.940+0000 7f0fd2082640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0fcc13f260 0x7f0fcc13f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:41.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.940+0000 7f0fd2082640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fcc0b2ab0 con 0x7f0fcc14dc70 2024-03-18T15:34:41.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.940+0000 7f0fd2082640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fcc0b0ea0 con 0x7f0fcc06d2d0 2024-03-18T15:34:41.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.940+0000 7f0fd2082640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f0fcc0728c0 con 0x7f0fcc13f260 2024-03-18T15:34:41.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.940+0000 7f0fcaffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0fcc13f260 0x7f0fcc13f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:41.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.940+0000 7f0fcaffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0fcc13f260 0x7f0fcc13f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33972/0 (socket says 172.21.15.5:33972) 2024-03-18T15:34:41.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.940+0000 7f0fcaffd640 1 -- 172.21.15.5:0/4237631924 learned_addr learned my addr 172.21.15.5:0/4237631924 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:41.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.940+0000 7f0fcbfff640 1 --1- 172.21.15.5:0/4237631924 >> v1:172.21.15.5:6789/0 conn(0x7f0fcc14dc70 0x7f0fcc150060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:39846/0 (socket says 172.21.15.5:39846) 2024-03-18T15:34:41.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.940+0000 7f0fcaffd640 1 -- 172.21.15.5:0/4237631924 >> v1:172.21.15.80:6789/0 conn(0x7f0fcc06d2d0 legacy=0x7f0fcc0b23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:41.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.940+0000 7f0fcaffd640 1 -- 172.21.15.5:0/4237631924 >> v1:172.21.15.5:6789/0 conn(0x7f0fcc14dc70 legacy=0x7f0fcc150060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:41.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.940+0000 7f0fcaffd640 1 -- 172.21.15.5:0/4237631924 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fcc06d6b0 con 0x7f0fcc13f260 2024-03-18T15:34:41.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.940+0000 7f0fca7fc640 1 -- 172.21.15.5:0/4237631924 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3525211629 0 0) 0x7f0fcc0b2ab0 con 0x7f0fcc14dc70 2024-03-18T15:34:41.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.940+0000 7f0fcaffd640 1 --2- 172.21.15.5:0/4237631924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fcc13f260 0x7f0fcc13f6a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0009870 tx=0x7f0fc001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d922424d4e01ea server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:41.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.940+0000 7f0fca7fc640 1 -- 172.21.15.5:0/4237631924 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc00b1070 con 0x7f0fcc13f260 2024-03-18T15:34:41.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.940+0000 7f0fca7fc640 1 -- 172.21.15.5:0/4237631924 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fc00ac620 con 0x7f0fcc13f260 2024-03-18T15:34:41.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.940+0000 7f0fca7fc640 1 -- 172.21.15.5:0/4237631924 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc00ac9f0 con 0x7f0fcc13f260 2024-03-18T15:34:41.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.940+0000 7f0fd2082640 1 -- 172.21.15.5:0/4237631924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fcc13f260 msgr2=0x7f0fcc13f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:41.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.940+0000 7f0fd2082640 1 --2- 172.21.15.5:0/4237631924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fcc13f260 0x7f0fcc13f6a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0009870 tx=0x7f0fc001c4d0 comp rx=0 tx=0).stop 2024-03-18T15:34:41.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.940+0000 7f0fd2082640 1 -- 172.21.15.5:0/4237631924 shutdown_connections 2024-03-18T15:34:41.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.940+0000 7f0fd2082640 1 --2- 172.21.15.5:0/4237631924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fcc13f260 0x7f0fcc13f6a0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:41.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.940+0000 7f0fd2082640 1 -- 172.21.15.5:0/4237631924 >> 172.21.15.5:0/4237631924 conn(0x7f0fcc0b5670 msgr2=0x7f0fcc0b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:41.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.940+0000 7f0fd2082640 1 -- 172.21.15.5:0/4237631924 shutdown_connections 2024-03-18T15:34:41.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.940+0000 7f0fd2082640 1 -- 172.21.15.5:0/4237631924 wait complete. 2024-03-18T15:34:41.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.944+0000 7f0fd2082640 1 Processor -- start 2024-03-18T15:34:41.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.944+0000 7f0fd2082640 1 -- start start 2024-03-18T15:34:41.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.944+0000 7f0fd2082640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0fcc06d2d0 0x7f0fcc1e1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:41.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.944+0000 7f0fd2082640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0fcc13f260 0x7f0fcc14ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:41.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.944+0000 7f0fd2082640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fcc14dc70 0x7f0fcc071580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:41.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.944+0000 7f0fd2082640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0fcc0b0960 con 0x7f0fcc14dc70 2024-03-18T15:34:41.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.944+0000 7f0fd2082640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0fcc0b07e0 con 0x7f0fcc06d2d0 2024-03-18T15:34:41.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.944+0000 7f0fd2082640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f0fcc0b5140 con 0x7f0fcc13f260 2024-03-18T15:34:41.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.944+0000 7f0fcbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fcc14dc70 0x7f0fcc071580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:41.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.944+0000 7f0fcbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fcc14dc70 0x7f0fcc071580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33980/0 (socket says 172.21.15.5:33980) 2024-03-18T15:34:41.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.944+0000 7f0fcbfff640 1 -- 172.21.15.5:0/404357016 learned_addr learned my addr 172.21.15.5:0/404357016 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:41.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.944+0000 7f0fcb7fe640 1 --2- 172.21.15.5:0/404357016 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0fcc06d2d0 0x7f0fcc1e1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:41.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.944+0000 7f0fcbfff640 1 -- 172.21.15.5:0/404357016 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0fcc13f260 msgr2=0x7f0fcc14ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:41.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.944+0000 7f0fcaffd640 1 --2- 172.21.15.5:0/404357016 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0fcc13f260 0x7f0fcc14ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:41.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.944+0000 7f0fcbfff640 1 --2- 172.21.15.5:0/404357016 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0fcc13f260 0x7f0fcc14ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:41.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.944+0000 7f0fcbfff640 1 -- 172.21.15.5:0/404357016 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0fcc06d2d0 msgr2=0x7f0fcc1e1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:41.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.944+0000 7f0fcbfff640 1 --2- 172.21.15.5:0/404357016 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0fcc06d2d0 0x7f0fcc1e1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:41.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.944+0000 7f0fcbfff640 1 -- 172.21.15.5:0/404357016 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fcc14e7b0 con 0x7f0fcc14dc70 2024-03-18T15:34:41.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.944+0000 7f0fcb7fe640 1 --2- 172.21.15.5:0/404357016 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0fcc06d2d0 0x7f0fcc1e1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:34:41.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.944+0000 7f0fcbfff640 1 --2- 172.21.15.5:0/404357016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fcc14dc70 0x7f0fcc071580 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f0fc400c7b0 tx=0x7f0fc400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:41.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.944+0000 7f0fb3fff640 1 -- 172.21.15.5:0/404357016 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc4018070 con 0x7f0fcc14dc70 2024-03-18T15:34:41.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.944+0000 7f0fb3fff640 1 -- 172.21.15.5:0/404357016 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fc400a640 con 0x7f0fcc14dc70 2024-03-18T15:34:41.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.944+0000 7f0fb3fff640 1 -- 172.21.15.5:0/404357016 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc400aa70 con 0x7f0fcc14dc70 2024-03-18T15:34:41.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.944+0000 7f0fd2082640 1 -- 172.21.15.5:0/404357016 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fcc14ea90 con 0x7f0fcc14dc70 2024-03-18T15:34:41.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.944+0000 7f0fd2082640 1 -- 172.21.15.5:0/404357016 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fcc145960 con 0x7f0fcc14dc70 2024-03-18T15:34:41.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.944+0000 7f0fd2082640 1 -- 172.21.15.5:0/404357016 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fcc04f550 con 0x7f0fcc14dc70 2024-03-18T15:34:41.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.948+0000 7f0fb3fff640 1 -- 172.21.15.5:0/404357016 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f0fc400ac10 con 0x7f0fcc14dc70 2024-03-18T15:34:41.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.948+0000 7f0fb3fff640 1 --2- 172.21.15.5:0/404357016 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0fb8067660 0x7f0fb8069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:41.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.948+0000 7f0fb3fff640 1 -- 172.21.15.5:0/404357016 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7f0fc4084c10 con 0x7f0fcc14dc70 2024-03-18T15:34:41.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.948+0000 7f0fcb7fe640 1 --2- 172.21.15.5:0/404357016 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0fb8067660 0x7f0fb8069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:41.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.948+0000 7f0fcb7fe640 1 --2- 172.21.15.5:0/404357016 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0fb8067660 0x7f0fb8069b20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc000f70 tx=0x7f0fbc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:41.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:41.952+0000 7f0fb3fff640 1 -- 172.21.15.5:0/404357016 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f0fc4055b80 con 0x7f0fcc14dc70 2024-03-18T15:34:42.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.108+0000 7f0fd2082640 1 -- 172.21.15.5:0/404357016 --> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0fcc12e9a0 con 0x7f0fb8067660 2024-03-18T15:34:42.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.108+0000 7f0fb3fff640 1 -- 172.21.15.5:0/404357016 <== mgr.4110 v2:172.21.15.5:6832/3362034860 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13486 (secure 0 0 0) 0x7f0fcc12e9a0 con 0x7f0fb8067660 2024-03-18T15:34:42.113 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:34:42.113 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-03-18T15:34:42.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.112+0000 7f0fd2082640 1 -- 172.21.15.5:0/404357016 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0fb8067660 msgr2=0x7f0fb8069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:42.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.112+0000 7f0fd2082640 1 --2- 172.21.15.5:0/404357016 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0fb8067660 0x7f0fb8069b20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc000f70 tx=0x7f0fbc009040 comp rx=0 tx=0).stop 2024-03-18T15:34:42.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.112+0000 7f0fd2082640 1 -- 172.21.15.5:0/404357016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fcc14dc70 msgr2=0x7f0fcc071580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:42.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.112+0000 7f0fd2082640 1 --2- 172.21.15.5:0/404357016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fcc14dc70 0x7f0fcc071580 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f0fc400c7b0 tx=0x7f0fc400cc70 comp rx=0 tx=0).stop 2024-03-18T15:34:42.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.112+0000 7f0fd2082640 1 -- 172.21.15.5:0/404357016 shutdown_connections 2024-03-18T15:34:42.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.112+0000 7f0fd2082640 1 --2- 172.21.15.5:0/404357016 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0fb8067660 0x7f0fb8069b20 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:42.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.112+0000 7f0fd2082640 1 --2- 172.21.15.5:0/404357016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fcc14dc70 0x7f0fcc071580 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:42.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.112+0000 7f0fd2082640 1 --2- 172.21.15.5:0/404357016 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0fcc13f260 0x7f0fcc14ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:42.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.112+0000 7f0fd2082640 1 --2- 172.21.15.5:0/404357016 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0fcc06d2d0 0x7f0fcc1e1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:42.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.112+0000 7f0fd2082640 1 -- 172.21.15.5:0/404357016 >> 172.21.15.5:0/404357016 conn(0x7f0fcc0b5670 msgr2=0x7f0fcc13cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:42.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.112+0000 7f0fd2082640 1 -- 172.21.15.5:0/404357016 shutdown_connections 2024-03-18T15:34:42.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.112+0000 7f0fd2082640 1 -- 172.21.15.5:0/404357016 wait complete. 2024-03-18T15:34:42.132 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":62,"stamp":"2024-03-18T15:34:40.901837+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":253348,"kb_used_data":3904,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754721372,"statfs":{"total":773094113280,"available":772834684928,"internally_reserved":0,"allocated":3997696,"data_stored":2268761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10891,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-54,"num_write_kb":-26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001763"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":190,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-18T15:34:37.208620+0000","last_change":"2024-03-18T15:33:30.766315+0000","last_active":"2024-03-18T15:34:37.208620+0000","last_peered":"2024-03-18T15:34:37.208620+0000","last_clean":"2024-03-18T15:34:37.208620+0000","last_became_active":"2024-03-18T15:33:30.763939+0000","last_became_peered":"2024-03-18T15:33:30.763939+0000","last_unstale":"2024-03-18T15:34:37.208620+0000","last_undegraded":"2024-03-18T15:34:37.208620+0000","last_fullsized":"2024-03-18T15:34:37.208620+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:29.229911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:29.229911+0000","last_clean_scrub_stamp":"2024-03-18T15:33:29.229911+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:10:21.120371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509458,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":356352,"data_stored":138617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542162,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343904,"statfs":{"total":96636764160,"available":96608157696,"internally_reserved":0,"allocated":937984,"data_stored":723072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542163,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542161,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35696,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336144,"statfs":{"total":96636764160,"available":96600211456,"internally_reserved":0,"allocated":364544,"data_stored":142729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542161,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35676,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336164,"statfs":{"total":96636764160,"available":96600231936,"internally_reserved":0,"allocated":344064,"data_stored":132684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542161,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35696,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336144,"statfs":{"total":96636764160,"available":96600211456,"internally_reserved":0,"allocated":364544,"data_stored":142729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542160,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36252,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335588,"statfs":{"total":96636764160,"available":96599642112,"internally_reserved":0,"allocated":933888,"data_stored":722982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-18T15:34:42.132 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-18T15:34:42.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.256+0000 7f3ebacbe640 1 Processor -- start 2024-03-18T15:34:42.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.256+0000 7f3ebacbe640 1 -- start start 2024-03-18T15:34:42.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.256+0000 7f3ebacbe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3eb40b5710 0x7f3eb414a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:42.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.256+0000 7f3ebacbe640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3eb414ae60 0x7f3eb4140ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:42.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.256+0000 7f3ebacbe640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3eb40b9d50 con 0x7f3eb4141430 2024-03-18T15:34:42.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.256+0000 7f3ebacbe640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f3eb40b4320 con 0x7f3eb414ae60 2024-03-18T15:34:42.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.256+0000 7f3ebacbe640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f3eb40b4570 con 0x7f3eb40b5710 2024-03-18T15:34:42.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.256+0000 7f3eb3fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3eb414ae60 0x7f3eb4140ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:42.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.256+0000 7f3eb3fff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f3eb414ae60 0x7f3eb4140ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33996/0 (socket says 172.21.15.5:33996) 2024-03-18T15:34:42.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.256+0000 7f3eb3fff640 1 -- 172.21.15.5:0/1486811232 learned_addr learned my addr 172.21.15.5:0/1486811232 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:42.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.256+0000 7f3eb8a5a640 1 --2- 172.21.15.5:0/1486811232 >> v2:172.21.15.80:3300/0 conn(0x7f3eb40b5710 0x7f3eb414a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:42.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.256+0000 7f3eb3fff640 1 -- 172.21.15.5:0/1486811232 >> v2:172.21.15.80:3300/0 conn(0x7f3eb40b5710 msgr2=0x7f3eb414a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:42.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.256+0000 7f3eb3fff640 1 --2- 172.21.15.5:0/1486811232 >> v2:172.21.15.80:3300/0 conn(0x7f3eb40b5710 0x7f3eb414a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:42.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.256+0000 7f3eb3fff640 1 -- 172.21.15.5:0/1486811232 >> v1:172.21.15.80:6789/0 conn(0x7f3eb4141430 legacy=0x7f3eb40b7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:42.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.256+0000 7f3eb3fff640 1 -- 172.21.15.5:0/1486811232 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eb40b4c30 con 0x7f3eb414ae60 2024-03-18T15:34:42.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.260+0000 7f3eb3fff640 1 --2- 172.21.15.5:0/1486811232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3eb414ae60 0x7f3eb4140ef0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3eb406c3e0 tx=0x7f3ea401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29e2c4f8a658c798 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:42.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.260+0000 7f3eb37fe640 1 -- 172.21.15.5:0/1486811232 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea40b0070 con 0x7f3eb414ae60 2024-03-18T15:34:42.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.260+0000 7f3eb37fe640 1 -- 172.21.15.5:0/1486811232 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ea40ab600 con 0x7f3eb414ae60 2024-03-18T15:34:42.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.260+0000 7f3eb37fe640 1 -- 172.21.15.5:0/1486811232 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea40ab9d0 con 0x7f3eb414ae60 2024-03-18T15:34:42.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.260+0000 7f3ebacbe640 1 -- 172.21.15.5:0/1486811232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3eb414ae60 msgr2=0x7f3eb4140ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:42.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.260+0000 7f3ebacbe640 1 --2- 172.21.15.5:0/1486811232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3eb414ae60 0x7f3eb4140ef0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3eb406c3e0 tx=0x7f3ea401c400 comp rx=0 tx=0).stop 2024-03-18T15:34:42.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.260+0000 7f3ebacbe640 1 -- 172.21.15.5:0/1486811232 shutdown_connections 2024-03-18T15:34:42.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.260+0000 7f3ebacbe640 1 --2- 172.21.15.5:0/1486811232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3eb414ae60 0x7f3eb4140ef0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:42.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.260+0000 7f3ebacbe640 1 --2- 172.21.15.5:0/1486811232 >> v2:172.21.15.80:3300/0 conn(0x7f3eb40b5710 0x7f3eb414a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:42.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.260+0000 7f3ebacbe640 1 -- 172.21.15.5:0/1486811232 >> 172.21.15.5:0/1486811232 conn(0x7f3eb40673c0 msgr2=0x7f3eb40677d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:42.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.260+0000 7f3ebacbe640 1 -- 172.21.15.5:0/1486811232 shutdown_connections 2024-03-18T15:34:42.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.260+0000 7f3ebacbe640 1 -- 172.21.15.5:0/1486811232 wait complete. 2024-03-18T15:34:42.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.260+0000 7f3ebacbe640 1 Processor -- start 2024-03-18T15:34:42.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.260+0000 7f3ebacbe640 1 -- start start 2024-03-18T15:34:42.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.260+0000 7f3ebacbe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3eb40b5710 0x7f3eb41e34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:42.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.260+0000 7f3ebacbe640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3eb4141430 0x7f3eb413b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:42.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.260+0000 7f3ebacbe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3eb414ae60 0x7f3eb4135c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:42.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.260+0000 7f3ebacbe640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb4141840 con 0x7f3eb40b5710 2024-03-18T15:34:42.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.260+0000 7f3ebacbe640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb414b2c0 con 0x7f3eb414ae60 2024-03-18T15:34:42.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.260+0000 7f3ebacbe640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f3eb41419c0 con 0x7f3eb4141430 2024-03-18T15:34:42.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.260+0000 7f3eb8a5a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3eb40b5710 0x7f3eb41e34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:42.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.260+0000 7f3eb8a5a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3eb40b5710 0x7f3eb41e34a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34012/0 (socket says 172.21.15.5:34012) 2024-03-18T15:34:42.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.260+0000 7f3eb8a5a640 1 -- 172.21.15.5:0/425430253 learned_addr learned my addr 172.21.15.5:0/425430253 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:42.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.260+0000 7f3eb925b640 1 --2- 172.21.15.5:0/425430253 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3eb414ae60 0x7f3eb4135c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:42.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.260+0000 7f3eb3fff640 1 --2- 172.21.15.5:0/425430253 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3eb4141430 0x7f3eb413b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:42.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.260+0000 7f3eb8a5a640 1 -- 172.21.15.5:0/425430253 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3eb4141430 msgr2=0x7f3eb413b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:42.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.260+0000 7f3eb8a5a640 1 --2- 172.21.15.5:0/425430253 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3eb4141430 0x7f3eb413b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:42.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.260+0000 7f3eb8a5a640 1 -- 172.21.15.5:0/425430253 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3eb414ae60 msgr2=0x7f3eb4135c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:42.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.260+0000 7f3eb8a5a640 1 --2- 172.21.15.5:0/425430253 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3eb414ae60 0x7f3eb4135c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:42.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.260+0000 7f3eb8a5a640 1 -- 172.21.15.5:0/425430253 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eb413bd60 con 0x7f3eb40b5710 2024-03-18T15:34:42.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.260+0000 7f3eb3fff640 1 --2- 172.21.15.5:0/425430253 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3eb4141430 0x7f3eb413b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:42.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.260+0000 7f3eb8a5a640 1 --2- 172.21.15.5:0/425430253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3eb40b5710 0x7f3eb41e34a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3ea800d620 tx=0x7f3ea800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:42.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.260+0000 7f3eb17fa640 1 -- 172.21.15.5:0/425430253 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea800b620 con 0x7f3eb40b5710 2024-03-18T15:34:42.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.260+0000 7f3eb17fa640 1 -- 172.21.15.5:0/425430253 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ea8019070 con 0x7f3eb40b5710 2024-03-18T15:34:42.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.260+0000 7f3ebacbe640 1 -- 172.21.15.5:0/425430253 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eb413a1b0 con 0x7f3eb40b5710 2024-03-18T15:34:42.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.260+0000 7f3eb17fa640 1 -- 172.21.15.5:0/425430253 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea800ba20 con 0x7f3eb40b5710 2024-03-18T15:34:42.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.260+0000 7f3ebacbe640 1 -- 172.21.15.5:0/425430253 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eb40b4010 con 0x7f3eb40b5710 2024-03-18T15:34:42.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.264+0000 7f3ebacbe640 1 -- 172.21.15.5:0/425430253 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3eb404f550 con 0x7f3eb40b5710 2024-03-18T15:34:42.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.264+0000 7f3eb17fa640 1 -- 172.21.15.5:0/425430253 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f3ea800bbc0 con 0x7f3eb40b5710 2024-03-18T15:34:42.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.264+0000 7f3eb17fa640 1 --2- 172.21.15.5:0/425430253 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3ea0067660 0x7f3ea0069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:42.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.264+0000 7f3eb17fa640 1 -- 172.21.15.5:0/425430253 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7f3ea8085a20 con 0x7f3eb40b5710 2024-03-18T15:34:42.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.264+0000 7f3eb3fff640 1 --2- 172.21.15.5:0/425430253 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3ea0067660 0x7f3ea0069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:42.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.264+0000 7f3eb3fff640 1 --2- 172.21.15.5:0/425430253 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3ea0067660 0x7f3ea0069b20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3ea40ad000 tx=0x7f3ea40a6000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:42.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.268+0000 7f3eb17fa640 1 -- 172.21.15.5:0/425430253 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f3ea8056990 con 0x7f3eb40b5710 2024-03-18T15:34:42.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.424+0000 7f3ebacbe640 1 -- 172.21.15.5:0/425430253 --> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3eb40b7140 con 0x7f3ea0067660 2024-03-18T15:34:42.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.428+0000 7f3eb17fa640 1 -- 172.21.15.5:0/425430253 <== mgr.4110 v2:172.21.15.5:6832/3362034860 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13486 (secure 0 0 0) 0x7f3eb40b7140 con 0x7f3ea0067660 2024-03-18T15:34:42.430 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:34:42.430 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-03-18T15:34:42.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.428+0000 7f3ebacbe640 1 -- 172.21.15.5:0/425430253 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3ea0067660 msgr2=0x7f3ea0069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:42.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.428+0000 7f3ebacbe640 1 --2- 172.21.15.5:0/425430253 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3ea0067660 0x7f3ea0069b20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3ea40ad000 tx=0x7f3ea40a6000 comp rx=0 tx=0).stop 2024-03-18T15:34:42.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.428+0000 7f3ebacbe640 1 -- 172.21.15.5:0/425430253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3eb40b5710 msgr2=0x7f3eb41e34a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:42.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.428+0000 7f3ebacbe640 1 --2- 172.21.15.5:0/425430253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3eb40b5710 0x7f3eb41e34a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3ea800d620 tx=0x7f3ea800dae0 comp rx=0 tx=0).stop 2024-03-18T15:34:42.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.428+0000 7f3ebacbe640 1 -- 172.21.15.5:0/425430253 shutdown_connections 2024-03-18T15:34:42.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.428+0000 7f3ebacbe640 1 --2- 172.21.15.5:0/425430253 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3ea0067660 0x7f3ea0069b20 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:42.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.428+0000 7f3ebacbe640 1 --2- 172.21.15.5:0/425430253 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3eb414ae60 0x7f3eb4135c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:42.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.428+0000 7f3ebacbe640 1 --2- 172.21.15.5:0/425430253 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3eb4141430 0x7f3eb413b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:42.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.428+0000 7f3ebacbe640 1 --2- 172.21.15.5:0/425430253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3eb40b5710 0x7f3eb41e34a0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:42.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.428+0000 7f3ebacbe640 1 -- 172.21.15.5:0/425430253 >> 172.21.15.5:0/425430253 conn(0x7f3eb40673c0 msgr2=0x7f3eb413cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:42.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.428+0000 7f3ebacbe640 1 -- 172.21.15.5:0/425430253 shutdown_connections 2024-03-18T15:34:42.433 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.428+0000 7f3ebacbe640 1 -- 172.21.15.5:0/425430253 wait complete. 2024-03-18T15:34:42.449 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":62,"stamp":"2024-03-18T15:34:40.901837+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":253348,"kb_used_data":3904,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754721372,"statfs":{"total":773094113280,"available":772834684928,"internally_reserved":0,"allocated":3997696,"data_stored":2268761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10891,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-54,"num_write_kb":-26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001763"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":190,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-18T15:34:37.208620+0000","last_change":"2024-03-18T15:33:30.766315+0000","last_active":"2024-03-18T15:34:37.208620+0000","last_peered":"2024-03-18T15:34:37.208620+0000","last_clean":"2024-03-18T15:34:37.208620+0000","last_became_active":"2024-03-18T15:33:30.763939+0000","last_became_peered":"2024-03-18T15:33:30.763939+0000","last_unstale":"2024-03-18T15:34:37.208620+0000","last_undegraded":"2024-03-18T15:34:37.208620+0000","last_fullsized":"2024-03-18T15:34:37.208620+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:29.229911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:29.229911+0000","last_clean_scrub_stamp":"2024-03-18T15:33:29.229911+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:10:21.120371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509458,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":356352,"data_stored":138617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542162,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343904,"statfs":{"total":96636764160,"available":96608157696,"internally_reserved":0,"allocated":937984,"data_stored":723072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542163,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542161,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35696,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336144,"statfs":{"total":96636764160,"available":96600211456,"internally_reserved":0,"allocated":364544,"data_stored":142729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542161,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35676,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336164,"statfs":{"total":96636764160,"available":96600231936,"internally_reserved":0,"allocated":344064,"data_stored":132684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542161,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35696,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336144,"statfs":{"total":96636764160,"available":96600211456,"internally_reserved":0,"allocated":364544,"data_stored":142729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542160,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36252,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335588,"statfs":{"total":96636764160,"available":96599642112,"internally_reserved":0,"allocated":933888,"data_stored":722982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-18T15:34:42.449 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-18T15:34:42.450 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-18T15:34:42.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.572+0000 7f1af76b6640 1 Processor -- start 2024-03-18T15:34:42.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.572+0000 7f1af76b6640 1 -- start start 2024-03-18T15:34:42.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.576+0000 7f1af76b6640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1af006d2d0 0x7f1af00b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:42.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.576+0000 7f1af76b6640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1af013f6a0 0x7f1af0145ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:42.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.576+0000 7f1af76b6640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1af013fb10 con 0x7f1af013f260 2024-03-18T15:34:42.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.576+0000 7f1af76b6640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f1af00728c0 con 0x7f1af013f6a0 2024-03-18T15:34:42.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.576+0000 7f1af76b6640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f1af00b0ea0 con 0x7f1af006d2d0 2024-03-18T15:34:42.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.576+0000 7f1af5452640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1af006d2d0 0x7f1af00b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:42.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.576+0000 7f1af5c53640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1af013f6a0 0x7f1af0145ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:42.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.576+0000 7f1af4c51640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f1af013f260 0x7f1af01453a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.5:35830/0 (socket says 172.21.15.5:35830) 2024-03-18T15:34:42.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.576+0000 7f1af5452640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1af006d2d0 0x7f1af00b23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:55794/0 (socket says 172.21.15.5:55794) 2024-03-18T15:34:42.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.576+0000 7f1af5c53640 1 --2- 172.21.15.5:0/2573820321 >> v2:172.21.15.80:3300/0 conn(0x7f1af013f6a0 0x7f1af0145ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:38420/0 (socket says 172.21.15.5:38420) 2024-03-18T15:34:42.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.576+0000 7f1af4c51640 1 -- 172.21.15.5:0/2573820321 learned_addr learned my addr 172.21.15.5:0/2573820321 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:42.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.576+0000 7f1af5452640 1 -- 172.21.15.5:0/2573820321 >> v2:172.21.15.80:3300/0 conn(0x7f1af013f6a0 msgr2=0x7f1af0145ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:42.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.576+0000 7f1af5452640 1 --2- 172.21.15.5:0/2573820321 >> v2:172.21.15.80:3300/0 conn(0x7f1af013f6a0 0x7f1af0145ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:42.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.576+0000 7f1af5452640 1 -- 172.21.15.5:0/2573820321 >> v1:172.21.15.80:6789/0 conn(0x7f1af013f260 legacy=0x7f1af01453a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:42.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.576+0000 7f1af5452640 1 -- 172.21.15.5:0/2573820321 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1af006d6b0 con 0x7f1af006d2d0 2024-03-18T15:34:42.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.580+0000 7f1af5452640 1 --2- 172.21.15.5:0/2573820321 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1af006d2d0 0x7f1af00b23a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1aec0097c0 tx=0x7f1aec01c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7033461c2bb4536d server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:42.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.580+0000 7f1ae7fff640 1 -- 172.21.15.5:0/2573820321 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aec0b1070 con 0x7f1af006d2d0 2024-03-18T15:34:42.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.580+0000 7f1ae7fff640 1 -- 172.21.15.5:0/2573820321 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1aec0a8050 con 0x7f1af006d2d0 2024-03-18T15:34:42.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.580+0000 7f1ae7fff640 1 -- 172.21.15.5:0/2573820321 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aec0ac860 con 0x7f1af006d2d0 2024-03-18T15:34:42.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.580+0000 7f1af76b6640 1 -- 172.21.15.5:0/2573820321 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1af006d2d0 msgr2=0x7f1af00b23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:42.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.580+0000 7f1af76b6640 1 --2- 172.21.15.5:0/2573820321 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1af006d2d0 0x7f1af00b23a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1aec0097c0 tx=0x7f1aec01c3e0 comp rx=0 tx=0).stop 2024-03-18T15:34:42.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.580+0000 7f1af76b6640 1 -- 172.21.15.5:0/2573820321 shutdown_connections 2024-03-18T15:34:42.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.580+0000 7f1af76b6640 1 --2- 172.21.15.5:0/2573820321 >> v2:172.21.15.80:3300/0 conn(0x7f1af013f6a0 0x7f1af0145ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:42.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.580+0000 7f1af76b6640 1 --2- 172.21.15.5:0/2573820321 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1af006d2d0 0x7f1af00b23a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:42.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.580+0000 7f1af76b6640 1 -- 172.21.15.5:0/2573820321 >> 172.21.15.5:0/2573820321 conn(0x7f1af00b5670 msgr2=0x7f1af00b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:42.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.580+0000 7f1af76b6640 1 -- 172.21.15.5:0/2573820321 shutdown_connections 2024-03-18T15:34:42.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.580+0000 7f1af76b6640 1 -- 172.21.15.5:0/2573820321 wait complete. 2024-03-18T15:34:42.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.580+0000 7f1af76b6640 1 Processor -- start 2024-03-18T15:34:42.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.580+0000 7f1af76b6640 1 -- start start 2024-03-18T15:34:42.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.580+0000 7f1af76b6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1af006d2d0 0x7f1af00ae530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:42.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.580+0000 7f1af76b6640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1af013f260 0x7f1af00b6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:42.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.580+0000 7f1af76b6640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1af013f6a0 0x7f1af00b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:42.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.580+0000 7f1af76b6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1af00b0960 con 0x7f1af006d2d0 2024-03-18T15:34:42.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.580+0000 7f1af76b6640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1af00b07e0 con 0x7f1af013f6a0 2024-03-18T15:34:42.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.580+0000 7f1af76b6640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f1af0072d50 con 0x7f1af013f260 2024-03-18T15:34:42.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.580+0000 7f1af5452640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1af006d2d0 0x7f1af00ae530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:42.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.580+0000 7f1af5452640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1af006d2d0 0x7f1af00ae530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34018/0 (socket says 172.21.15.5:34018) 2024-03-18T15:34:42.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.580+0000 7f1af5452640 1 -- 172.21.15.5:0/2980056437 learned_addr learned my addr 172.21.15.5:0/2980056437 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:42.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.580+0000 7f1af4c51640 1 --2- 172.21.15.5:0/2980056437 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1af013f260 0x7f1af00b6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:42.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.580+0000 7f1af5c53640 1 --2- 172.21.15.5:0/2980056437 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1af013f6a0 0x7f1af00b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:42.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.584+0000 7f1af5452640 1 -- 172.21.15.5:0/2980056437 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1af013f260 msgr2=0x7f1af00b6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:42.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.584+0000 7f1af5452640 1 --2- 172.21.15.5:0/2980056437 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1af013f260 0x7f1af00b6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:42.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.584+0000 7f1af5452640 1 -- 172.21.15.5:0/2980056437 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1af013f6a0 msgr2=0x7f1af00b72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:42.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.584+0000 7f1af5452640 1 --2- 172.21.15.5:0/2980056437 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1af013f6a0 0x7f1af00b72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:42.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.584+0000 7f1af5452640 1 -- 172.21.15.5:0/2980056437 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1af00b8700 con 0x7f1af006d2d0 2024-03-18T15:34:42.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.584+0000 7f1af5452640 1 --2- 172.21.15.5:0/2980056437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1af006d2d0 0x7f1af00ae530 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1aec01a010 tx=0x7f1aec019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:42.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.584+0000 7f1ae5ffb640 1 -- 172.21.15.5:0/2980056437 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aec0b9070 con 0x7f1af006d2d0 2024-03-18T15:34:42.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.584+0000 7f1ae5ffb640 1 -- 172.21.15.5:0/2980056437 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1aec0acdf0 con 0x7f1af006d2d0 2024-03-18T15:34:42.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.584+0000 7f1af76b6640 1 -- 172.21.15.5:0/2980056437 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1af0144c30 con 0x7f1af006d2d0 2024-03-18T15:34:42.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.584+0000 7f1ae5ffb640 1 -- 172.21.15.5:0/2980056437 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aec0b1040 con 0x7f1af006d2d0 2024-03-18T15:34:42.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.584+0000 7f1af76b6640 1 -- 172.21.15.5:0/2980056437 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1af00b7aa0 con 0x7f1af006d2d0 2024-03-18T15:34:42.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.584+0000 7f1af76b6640 1 -- 172.21.15.5:0/2980056437 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1af004f550 con 0x7f1af006d2d0 2024-03-18T15:34:42.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.584+0000 7f1ae5ffb640 1 -- 172.21.15.5:0/2980056437 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f1aec0bc070 con 0x7f1af006d2d0 2024-03-18T15:34:42.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.584+0000 7f1ae5ffb640 1 --2- 172.21.15.5:0/2980056437 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f1ad4067660 0x7f1ad4069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:42.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.584+0000 7f1ae5ffb640 1 -- 172.21.15.5:0/2980056437 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7f1aec11f330 con 0x7f1af006d2d0 2024-03-18T15:34:42.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.584+0000 7f1af4c51640 1 --2- 172.21.15.5:0/2980056437 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f1ad4067660 0x7f1ad4069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:42.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.584+0000 7f1af4c51640 1 --2- 172.21.15.5:0/2980056437 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f1ad4067660 0x7f1ad4069b20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1af00aee00 tx=0x7f1ae0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:42.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.592+0000 7f1ae5ffb640 1 -- 172.21.15.5:0/2980056437 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f1aec016090 con 0x7f1af006d2d0 2024-03-18T15:34:42.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.748+0000 7f1af76b6640 1 -- 172.21.15.5:0/2980056437 --> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1af0144a80 con 0x7f1ad4067660 2024-03-18T15:34:42.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.748+0000 7f1ae5ffb640 1 -- 172.21.15.5:0/2980056437 <== mgr.4110 v2:172.21.15.5:6832/3362034860 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13486 (secure 0 0 0) 0x7f1af0144a80 con 0x7f1ad4067660 2024-03-18T15:34:42.753 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:34:42.753 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-03-18T15:34:42.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.752+0000 7f1af76b6640 1 -- 172.21.15.5:0/2980056437 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f1ad4067660 msgr2=0x7f1ad4069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:42.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.752+0000 7f1af76b6640 1 --2- 172.21.15.5:0/2980056437 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f1ad4067660 0x7f1ad4069b20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1af00aee00 tx=0x7f1ae0009040 comp rx=0 tx=0).stop 2024-03-18T15:34:42.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.752+0000 7f1af76b6640 1 -- 172.21.15.5:0/2980056437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1af006d2d0 msgr2=0x7f1af00ae530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:42.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.752+0000 7f1af76b6640 1 --2- 172.21.15.5:0/2980056437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1af006d2d0 0x7f1af00ae530 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1aec01a010 tx=0x7f1aec019180 comp rx=0 tx=0).stop 2024-03-18T15:34:42.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.752+0000 7f1af76b6640 1 -- 172.21.15.5:0/2980056437 shutdown_connections 2024-03-18T15:34:42.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.752+0000 7f1af76b6640 1 --2- 172.21.15.5:0/2980056437 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f1ad4067660 0x7f1ad4069b20 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:42.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.752+0000 7f1af76b6640 1 --2- 172.21.15.5:0/2980056437 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1af013f6a0 0x7f1af00b72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:42.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.752+0000 7f1af76b6640 1 --2- 172.21.15.5:0/2980056437 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f1af013f260 0x7f1af00b6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:42.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.752+0000 7f1af76b6640 1 --2- 172.21.15.5:0/2980056437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1af006d2d0 0x7f1af00ae530 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:42.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.752+0000 7f1af76b6640 1 -- 172.21.15.5:0/2980056437 >> 172.21.15.5:0/2980056437 conn(0x7f1af00b5670 msgr2=0x7f1af00b29b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:42.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.752+0000 7f1af76b6640 1 -- 172.21.15.5:0/2980056437 shutdown_connections 2024-03-18T15:34:42.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.752+0000 7f1af76b6640 1 -- 172.21.15.5:0/2980056437 wait complete. 2024-03-18T15:34:42.772 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":62,"stamp":"2024-03-18T15:34:40.901837+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":253348,"kb_used_data":3904,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754721372,"statfs":{"total":773094113280,"available":772834684928,"internally_reserved":0,"allocated":3997696,"data_stored":2268761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10891,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-54,"num_write_kb":-26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001763"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":190,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-03-18T15:34:37.208620+0000","last_change":"2024-03-18T15:33:30.766315+0000","last_active":"2024-03-18T15:34:37.208620+0000","last_peered":"2024-03-18T15:34:37.208620+0000","last_clean":"2024-03-18T15:34:37.208620+0000","last_became_active":"2024-03-18T15:33:30.763939+0000","last_became_peered":"2024-03-18T15:33:30.763939+0000","last_unstale":"2024-03-18T15:34:37.208620+0000","last_undegraded":"2024-03-18T15:34:37.208620+0000","last_fullsized":"2024-03-18T15:34:37.208620+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:29.229911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:29.229911+0000","last_clean_scrub_stamp":"2024-03-18T15:33:29.229911+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:10:21.120371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509458,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":133230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509458,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27372,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344468,"statfs":{"total":96636764160,"available":96608735232,"internally_reserved":0,"allocated":356352,"data_stored":138617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542162,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343904,"statfs":{"total":96636764160,"available":96608157696,"internally_reserved":0,"allocated":937984,"data_stored":723072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542163,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":132718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542161,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35696,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336144,"statfs":{"total":96636764160,"available":96600211456,"internally_reserved":0,"allocated":364544,"data_stored":142729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542161,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35676,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336164,"statfs":{"total":96636764160,"available":96600231936,"internally_reserved":0,"allocated":344064,"data_stored":132684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542161,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35696,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336144,"statfs":{"total":96636764160,"available":96600211456,"internally_reserved":0,"allocated":364544,"data_stored":142729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542160,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36252,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335588,"statfs":{"total":96636764160,"available":96599642112,"internally_reserved":0,"allocated":933888,"data_stored":722982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-18T15:34:42.772 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-18T15:34:42.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.892+0000 7fdd13e68640 1 Processor -- start 2024-03-18T15:34:42.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.892+0000 7fdd13e68640 1 -- start start 2024-03-18T15:34:42.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.896+0000 7fdd13e68640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fdd0c13f260 0x7fdd0c13f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:42.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.896+0000 7fdd13e68640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdd0c14dc70 0x7fdd0c150060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:42.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.896+0000 7fdd13e68640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd0c0b2ab0 con 0x7fdd0c06d2d0 2024-03-18T15:34:42.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.896+0000 7fdd13e68640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fdd0c0728c0 con 0x7fdd0c13f260 2024-03-18T15:34:42.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.896+0000 7fdd13e68640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fdd0c0b0ea0 con 0x7fdd0c14dc70 2024-03-18T15:34:42.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.896+0000 7fdd11403640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fdd0c13f260 0x7fdd0c13f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:42.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.896+0000 7fdd12405640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdd0c14dc70 0x7fdd0c150060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:42.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.896+0000 7fdd11403640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fdd0c13f260 0x7fdd0c13f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.5:38432/0 (socket says 172.21.15.5:38432) 2024-03-18T15:34:42.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.896+0000 7fdd11403640 1 -- 172.21.15.5:0/2148899752 learned_addr learned my addr 172.21.15.5:0/2148899752 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:42.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.896+0000 7fdd11c04640 1 --1- 172.21.15.5:0/2148899752 >> v1:172.21.15.80:6789/0 conn(0x7fdd0c06d2d0 0x7fdd0c0b23a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.5:35834/0 (socket says 172.21.15.5:35834) 2024-03-18T15:34:42.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.896+0000 7fdd12405640 1 -- 172.21.15.5:0/2148899752 >> v2:172.21.15.80:3300/0 conn(0x7fdd0c13f260 msgr2=0x7fdd0c13f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:42.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.896+0000 7fdd12405640 1 --2- 172.21.15.5:0/2148899752 >> v2:172.21.15.80:3300/0 conn(0x7fdd0c13f260 0x7fdd0c13f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:42.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.896+0000 7fdd12405640 1 -- 172.21.15.5:0/2148899752 >> v1:172.21.15.80:6789/0 conn(0x7fdd0c06d2d0 legacy=0x7fdd0c0b23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:42.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.896+0000 7fdd12405640 1 -- 172.21.15.5:0/2148899752 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd0c06d6b0 con 0x7fdd0c14dc70 2024-03-18T15:34:42.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.896+0000 7fdd11403640 1 --2- 172.21.15.5:0/2148899752 >> v2:172.21.15.80:3300/0 conn(0x7fdd0c13f260 0x7fdd0c13f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:34:42.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.900+0000 7fdd12405640 1 --2- 172.21.15.5:0/2148899752 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdd0c14dc70 0x7fdd0c150060 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdd04009d90 tx=0x7fdd0401c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=505db8e930b80762 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:42.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.900+0000 7fdd10c02640 1 -- 172.21.15.5:0/2148899752 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd040b2070 con 0x7fdd0c14dc70 2024-03-18T15:34:42.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.900+0000 7fdd10c02640 1 -- 172.21.15.5:0/2148899752 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd040a57a0 con 0x7fdd0c14dc70 2024-03-18T15:34:42.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.900+0000 7fdd10c02640 1 -- 172.21.15.5:0/2148899752 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd040a5b70 con 0x7fdd0c14dc70 2024-03-18T15:34:42.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.900+0000 7fdd13e68640 1 -- 172.21.15.5:0/2148899752 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdd0c14dc70 msgr2=0x7fdd0c150060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:42.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.900+0000 7fdd13e68640 1 --2- 172.21.15.5:0/2148899752 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdd0c14dc70 0x7fdd0c150060 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdd04009d90 tx=0x7fdd0401c8e0 comp rx=0 tx=0).stop 2024-03-18T15:34:42.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.900+0000 7fdd13e68640 1 -- 172.21.15.5:0/2148899752 shutdown_connections 2024-03-18T15:34:42.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.900+0000 7fdd13e68640 1 --2- 172.21.15.5:0/2148899752 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdd0c14dc70 0x7fdd0c150060 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:42.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.900+0000 7fdd13e68640 1 --2- 172.21.15.5:0/2148899752 >> v2:172.21.15.80:3300/0 conn(0x7fdd0c13f260 0x7fdd0c13f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:42.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.900+0000 7fdd13e68640 1 -- 172.21.15.5:0/2148899752 >> 172.21.15.5:0/2148899752 conn(0x7fdd0c0b5670 msgr2=0x7fdd0c0b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:42.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.900+0000 7fdd13e68640 1 -- 172.21.15.5:0/2148899752 shutdown_connections 2024-03-18T15:34:42.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.900+0000 7fdd13e68640 1 -- 172.21.15.5:0/2148899752 wait complete. 2024-03-18T15:34:42.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.900+0000 7fdd13e68640 1 Processor -- start 2024-03-18T15:34:42.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.900+0000 7fdd13e68640 1 -- start start 2024-03-18T15:34:42.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.900+0000 7fdd13e68640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdd0c06d2d0 0x7fdd0c13aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:42.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.900+0000 7fdd13e68640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd0c13f260 0x7fdd0c1df890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:42.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.900+0000 7fdd13e68640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd0c14dc70 0x7fdd0c1dfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:42.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.900+0000 7fdd13e68640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c0b0960 con 0x7fdd0c13f260 2024-03-18T15:34:42.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.900+0000 7fdd13e68640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c0b07e0 con 0x7fdd0c14dc70 2024-03-18T15:34:42.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.900+0000 7fdd13e68640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fdd0c0b5140 con 0x7fdd0c06d2d0 2024-03-18T15:34:42.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.900+0000 7fdd11403640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd0c13f260 0x7fdd0c1df890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:42.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.900+0000 7fdd11403640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd0c13f260 0x7fdd0c1df890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34022/0 (socket says 172.21.15.5:34022) 2024-03-18T15:34:42.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.900+0000 7fdd11403640 1 -- 172.21.15.5:0/4072300062 learned_addr learned my addr 172.21.15.5:0/4072300062 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:42.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.900+0000 7fdd11c04640 1 --2- 172.21.15.5:0/4072300062 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdd0c06d2d0 0x7fdd0c13aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:42.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.900+0000 7fdd12405640 1 --2- 172.21.15.5:0/4072300062 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd0c14dc70 0x7fdd0c1dfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:42.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.900+0000 7fdd11403640 1 -- 172.21.15.5:0/4072300062 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdd0c06d2d0 msgr2=0x7fdd0c13aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:42.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.900+0000 7fdd11403640 1 --2- 172.21.15.5:0/4072300062 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdd0c06d2d0 0x7fdd0c13aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:42.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.900+0000 7fdd11403640 1 -- 172.21.15.5:0/4072300062 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd0c14dc70 msgr2=0x7fdd0c1dfdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:42.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.900+0000 7fdd11403640 1 --2- 172.21.15.5:0/4072300062 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd0c14dc70 0x7fdd0c1dfdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:42.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.900+0000 7fdd11403640 1 -- 172.21.15.5:0/4072300062 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd0c144930 con 0x7fdd0c13f260 2024-03-18T15:34:42.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.900+0000 7fdd11403640 1 --2- 172.21.15.5:0/4072300062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd0c13f260 0x7fdd0c1df890 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc00e9d0 tx=0x7fdcfc00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:42.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.904+0000 7fdd027fc640 1 -- 172.21.15.5:0/4072300062 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc017070 con 0x7fdd0c13f260 2024-03-18T15:34:42.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.904+0000 7fdd027fc640 1 -- 172.21.15.5:0/4072300062 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcfc00ca00 con 0x7fdd0c13f260 2024-03-18T15:34:42.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.904+0000 7fdd13e68640 1 -- 172.21.15.5:0/4072300062 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd0c071bc0 con 0x7fdd0c13f260 2024-03-18T15:34:42.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.904+0000 7fdd027fc640 1 -- 172.21.15.5:0/4072300062 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcfc00cdb0 con 0x7fdd0c13f260 2024-03-18T15:34:42.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.904+0000 7fdd13e68640 1 -- 172.21.15.5:0/4072300062 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd0c070b30 con 0x7fdd0c13f260 2024-03-18T15:34:42.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.904+0000 7fdd13e68640 1 -- 172.21.15.5:0/4072300062 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd0c04f550 con 0x7fdd0c13f260 2024-03-18T15:34:42.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.904+0000 7fdd027fc640 1 -- 172.21.15.5:0/4072300062 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fdcfc01a070 con 0x7fdd0c13f260 2024-03-18T15:34:42.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.904+0000 7fdd027fc640 1 --2- 172.21.15.5:0/4072300062 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fdcf0067660 0x7fdcf0069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:42.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.904+0000 7fdd027fc640 1 -- 172.21.15.5:0/4072300062 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7fdcfc085b30 con 0x7fdd0c13f260 2024-03-18T15:34:42.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.904+0000 7fdd11c04640 1 --2- 172.21.15.5:0/4072300062 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fdcf0067660 0x7fdcf0069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:42.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.904+0000 7fdd11c04640 1 --2- 172.21.15.5:0/4072300062 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fdcf0067660 0x7fdcf0069b20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fdd08000f70 tx=0x7fdd08007cf0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:42.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:42.908+0000 7fdd027fc640 1 -- 172.21.15.5:0/4072300062 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fdcfc056c20 con 0x7fdd0c13f260 2024-03-18T15:34:43.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:43.072+0000 7fdd13e68640 1 -- 172.21.15.5:0/4072300062 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdd0c13cc20 con 0x7fdd0c13f260 2024-03-18T15:34:43.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:43.072+0000 7fdd027fc640 1 -- 172.21.15.5:0/4072300062 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+12862 (secure 0 0 0) 0x7fdcfc056520 con 0x7fdd0c13f260 2024-03-18T15:34:43.075 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:34:43.075 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":33,"fsid":"ef54fe03-5aa7-4eb2-9b1e-66c9cc345b94","created":"2024-03-18T15:33:18.086614+0000","modified":"2024-03-18T15:34:40.214290+0000","last_up_change":"2024-03-18T15:33:29.229911+0000","last_in_change":"2024-03-18T15:33:20.323840+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-18T15:33:28.963186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"df1c754b-af59-4d16-8919-fbb6386b3c48","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6804","nonce":976264754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6808","nonce":976264754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6815","nonce":976264754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6811","nonce":976264754},{"type":"v1","addr":"172.21.15.5:6813","nonce":976264754}]},"public_addr":"172.21.15.5:6804/976264754","cluster_addr":"172.21.15.5:6808/976264754","heartbeat_back_addr":"172.21.15.5:6815/976264754","heartbeat_front_addr":"172.21.15.5:6813/976264754","state":["exists","up"]},{"osd":1,"uuid":"b9a08b5e-8af4-4144-bdba-d69ed1ac627a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6801","nonce":3901761126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6803","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6805","nonce":3901761126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6812","nonce":3901761126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6807","nonce":3901761126},{"type":"v1","addr":"172.21.15.5:6809","nonce":3901761126}]},"public_addr":"172.21.15.5:6801/3901761126","cluster_addr":"172.21.15.5:6805/3901761126","heartbeat_back_addr":"172.21.15.5:6812/3901761126","heartbeat_front_addr":"172.21.15.5:6809/3901761126","state":["exists","up"]},{"osd":2,"uuid":"c3e8bb76-e382-439d-b380-62ef92f1255a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6817","nonce":502968448}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6819","nonce":502968448}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6823","nonce":502968448}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":502968448},{"type":"v1","addr":"172.21.15.5:6821","nonce":502968448}]},"public_addr":"172.21.15.5:6817/502968448","cluster_addr":"172.21.15.5:6819/502968448","heartbeat_back_addr":"172.21.15.5:6823/502968448","heartbeat_front_addr":"172.21.15.5:6821/502968448","state":["exists","up"]},{"osd":3,"uuid":"e8c94625-f2cd-4eba-b53d-2e7356e1c02a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6825","nonce":2425932735}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6827","nonce":2425932735}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6831","nonce":2425932735}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":2425932735},{"type":"v1","addr":"172.21.15.5:6829","nonce":2425932735}]},"public_addr":"172.21.15.5:6825/2425932735","cluster_addr":"172.21.15.5:6827/2425932735","heartbeat_back_addr":"172.21.15.5:6831/2425932735","heartbeat_front_addr":"172.21.15.5:6829/2425932735","state":["exists","up"]},{"osd":4,"uuid":"0099dd52-6265-4d32-82e4-3ef16e73add0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6817","nonce":976514285}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6819","nonce":976514285}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6823","nonce":976514285}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6820","nonce":976514285},{"type":"v1","addr":"172.21.15.80:6821","nonce":976514285}]},"public_addr":"172.21.15.80:6817/976514285","cluster_addr":"172.21.15.80:6819/976514285","heartbeat_back_addr":"172.21.15.80:6823/976514285","heartbeat_front_addr":"172.21.15.80:6821/976514285","state":["exists","up"]},{"osd":5,"uuid":"ed10c84f-8c39-41cc-9b64-7c482a94a48e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6824","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6825","nonce":627153447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6826","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6827","nonce":627153447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6831","nonce":627153447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":627153447},{"type":"v1","addr":"172.21.15.80:6829","nonce":627153447}]},"public_addr":"172.21.15.80:6825/627153447","cluster_addr":"172.21.15.80:6827/627153447","heartbeat_back_addr":"172.21.15.80:6831/627153447","heartbeat_front_addr":"172.21.15.80:6829/627153447","state":["exists","up"]},{"osd":6,"uuid":"c7a3538a-7048-4fba-8781-b440f03c25db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6803","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6805","nonce":488342527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6807","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6809","nonce":488342527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6814","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6815","nonce":488342527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6811","nonce":488342527},{"type":"v1","addr":"172.21.15.80:6813","nonce":488342527}]},"public_addr":"172.21.15.80:6805/488342527","cluster_addr":"172.21.15.80:6809/488342527","heartbeat_back_addr":"172.21.15.80:6815/488342527","heartbeat_front_addr":"172.21.15.80:6813/488342527","state":["exists","up"]},{"osd":7,"uuid":"f9418104-3a97-4110-bc3a-1f8151d0f757","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6801","nonce":982831666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6804","nonce":982831666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6810","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6812","nonce":982831666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":982831666},{"type":"v1","addr":"172.21.15.80:6808","nonce":982831666}]},"public_addr":"172.21.15.80:6801/982831666","cluster_addr":"172.21.15.80:6804/982831666","heartbeat_back_addr":"172.21.15.80:6812/982831666","heartbeat_front_addr":"172.21.15.80:6808/982831666","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.970828+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.858116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.987556+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.864603+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.756293+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989597+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.989323+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-18T15:33:25.591502+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6835/2235154713":"2024-03-19T15:34:37.202231+0000","172.21.15.5:6834/2235154713":"2024-03-19T15:34:37.202231+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-18T15:34:43.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:43.076+0000 7fdd13e68640 1 -- 172.21.15.5:0/4072300062 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fdcf0067660 msgr2=0x7fdcf0069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:43.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:43.076+0000 7fdd13e68640 1 --2- 172.21.15.5:0/4072300062 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fdcf0067660 0x7fdcf0069b20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fdd08000f70 tx=0x7fdd08007cf0 comp rx=0 tx=0).stop 2024-03-18T15:34:43.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:43.076+0000 7fdd13e68640 1 -- 172.21.15.5:0/4072300062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd0c13f260 msgr2=0x7fdd0c1df890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:43.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:43.076+0000 7fdd13e68640 1 --2- 172.21.15.5:0/4072300062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd0c13f260 0x7fdd0c1df890 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc00e9d0 tx=0x7fdcfc00ee90 comp rx=0 tx=0).stop 2024-03-18T15:34:43.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:43.076+0000 7fdd13e68640 1 -- 172.21.15.5:0/4072300062 shutdown_connections 2024-03-18T15:34:43.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:43.076+0000 7fdd13e68640 1 --2- 172.21.15.5:0/4072300062 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fdcf0067660 0x7fdcf0069b20 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:43.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:43.076+0000 7fdd13e68640 1 --2- 172.21.15.5:0/4072300062 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd0c14dc70 0x7fdd0c1dfdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:43.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:43.076+0000 7fdd13e68640 1 --2- 172.21.15.5:0/4072300062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd0c13f260 0x7fdd0c1df890 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:43.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:43.076+0000 7fdd13e68640 1 --2- 172.21.15.5:0/4072300062 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fdd0c06d2d0 0x7fdd0c13aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:43.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:43.076+0000 7fdd13e68640 1 -- 172.21.15.5:0/4072300062 >> 172.21.15.5:0/4072300062 conn(0x7fdd0c0b5670 msgr2=0x7fdd0c0b5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:43.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:43.076+0000 7fdd13e68640 1 -- 172.21.15.5:0/4072300062 shutdown_connections 2024-03-18T15:34:43.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:43.076+0000 7fdd13e68640 1 -- 172.21.15.5:0/4072300062 wait complete. 2024-03-18T15:34:44.095 INFO:tasks.ceph:Scrubbing osd.0 2024-03-18T15:34:44.096 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-03-18T15:34:44.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.232+0000 7f0c9d052640 1 Processor -- start 2024-03-18T15:34:44.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.232+0000 7f0c9d052640 1 -- start start 2024-03-18T15:34:44.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.232+0000 7f0c9d052640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0c9806d300 0x7f0c980b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:44.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.232+0000 7f0c9d052640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0c980b1c00 0x7f0c98144160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:44.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.232+0000 7f0c9d052640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c9806d6e0 con 0x7f0c98071f70 2024-03-18T15:34:44.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.232+0000 7f0c9d052640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f0c9813e9c0 con 0x7f0c9806d300 2024-03-18T15:34:44.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.232+0000 7f0c9d052640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f0c980b2010 con 0x7f0c980b1c00 2024-03-18T15:34:44.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.232+0000 7f0c96d9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0c9806d300 0x7f0c980b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:44.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.232+0000 7f0c96d9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0c9806d300 0x7f0c980b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34036/0 (socket says 172.21.15.5:34036) 2024-03-18T15:34:44.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.232+0000 7f0c96d9d640 1 -- 172.21.15.5:0/63948315 learned_addr learned my addr 172.21.15.5:0/63948315 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:44.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.232+0000 7f0c9659c640 1 --2- 172.21.15.5:0/63948315 >> v2:172.21.15.80:3300/0 conn(0x7f0c980b1c00 0x7f0c98144160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:44.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.232+0000 7f0c96d9d640 1 -- 172.21.15.5:0/63948315 >> v2:172.21.15.80:3300/0 conn(0x7f0c980b1c00 msgr2=0x7f0c98144160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:44.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.232+0000 7f0c96d9d640 1 --2- 172.21.15.5:0/63948315 >> v2:172.21.15.80:3300/0 conn(0x7f0c980b1c00 0x7f0c98144160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:44.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.232+0000 7f0c96d9d640 1 -- 172.21.15.5:0/63948315 >> v1:172.21.15.80:6789/0 conn(0x7f0c98071f70 legacy=0x7f0c981476d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:44.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.232+0000 7f0c96d9d640 1 -- 172.21.15.5:0/63948315 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c980b1700 con 0x7f0c9806d300 2024-03-18T15:34:44.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.236+0000 7f0c96d9d640 1 --2- 172.21.15.5:0/63948315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c9806d300 0x7f0c980b4cf0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0c9806db00 tx=0x7f0c8801c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2353c3e19373ab38 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:44.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.236+0000 7f0c95d9b640 1 -- 172.21.15.5:0/63948315 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c880a5790 con 0x7f0c9806d300 2024-03-18T15:34:44.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.236+0000 7f0c95d9b640 1 -- 172.21.15.5:0/63948315 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c880a5930 con 0x7f0c9806d300 2024-03-18T15:34:44.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.236+0000 7f0c95d9b640 1 -- 172.21.15.5:0/63948315 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c880a5d00 con 0x7f0c9806d300 2024-03-18T15:34:44.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.236+0000 7f0c9d052640 1 -- 172.21.15.5:0/63948315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c9806d300 msgr2=0x7f0c980b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:44.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.236+0000 7f0c9d052640 1 --2- 172.21.15.5:0/63948315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c9806d300 0x7f0c980b4cf0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0c9806db00 tx=0x7f0c8801c8d0 comp rx=0 tx=0).stop 2024-03-18T15:34:44.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.236+0000 7f0c9d052640 1 -- 172.21.15.5:0/63948315 shutdown_connections 2024-03-18T15:34:44.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.236+0000 7f0c9d052640 1 --2- 172.21.15.5:0/63948315 >> v2:172.21.15.80:3300/0 conn(0x7f0c980b1c00 0x7f0c98144160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:44.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.236+0000 7f0c9d052640 1 --2- 172.21.15.5:0/63948315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c9806d300 0x7f0c980b4cf0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:44.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.236+0000 7f0c9d052640 1 -- 172.21.15.5:0/63948315 >> 172.21.15.5:0/63948315 conn(0x7f0c980b5670 msgr2=0x7f0c980b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:44.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.236+0000 7f0c9d052640 1 -- 172.21.15.5:0/63948315 shutdown_connections 2024-03-18T15:34:44.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.236+0000 7f0c9d052640 1 -- 172.21.15.5:0/63948315 wait complete. 2024-03-18T15:34:44.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.236+0000 7f0c9d052640 1 Processor -- start 2024-03-18T15:34:44.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.240+0000 7f0c9d052640 1 -- start start 2024-03-18T15:34:44.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.240+0000 7f0c9d052640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c9806d300 0x7f0c981457a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:44.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.240+0000 7f0c9d052640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c98071f70 0x7f0c981e00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:44.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.240+0000 7f0c9d052640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0c981e0630 0x7f0c981e3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:44.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.240+0000 7f0c9d052640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0c98072790 con 0x7f0c98071f70 2024-03-18T15:34:44.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.240+0000 7f0c9d052640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0c980b52f0 con 0x7f0c9806d300 2024-03-18T15:34:44.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.240+0000 7f0c9d052640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f0c981446a0 con 0x7f0c981e0630 2024-03-18T15:34:44.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.240+0000 7f0c9659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c98071f70 0x7f0c981e00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:44.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.240+0000 7f0c9659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c98071f70 0x7f0c981e00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34040/0 (socket says 172.21.15.5:34040) 2024-03-18T15:34:44.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.240+0000 7f0c9659c640 1 -- 172.21.15.5:0/2870187682 learned_addr learned my addr 172.21.15.5:0/2870187682 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:44.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.240+0000 7f0c9759e640 1 --2- 172.21.15.5:0/2870187682 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0c981e0630 0x7f0c981e3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:44.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.240+0000 7f0c96d9d640 1 --2- 172.21.15.5:0/2870187682 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c9806d300 0x7f0c981457a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:44.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.240+0000 7f0c9659c640 1 -- 172.21.15.5:0/2870187682 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0c981e0630 msgr2=0x7f0c981e3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:44.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.240+0000 7f0c9659c640 1 --2- 172.21.15.5:0/2870187682 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0c981e0630 0x7f0c981e3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:44.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.240+0000 7f0c9659c640 1 -- 172.21.15.5:0/2870187682 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c9806d300 msgr2=0x7f0c981457a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:44.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.240+0000 7f0c9659c640 1 --2- 172.21.15.5:0/2870187682 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c9806d300 0x7f0c981457a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:44.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.240+0000 7f0c9659c640 1 -- 172.21.15.5:0/2870187682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c98146800 con 0x7f0c98071f70 2024-03-18T15:34:44.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.240+0000 7f0c9659c640 1 --2- 172.21.15.5:0/2870187682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c98071f70 0x7f0c981e00f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0c8000d5f0 tx=0x7f0c8000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:44.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.240+0000 7f0c8f7fe640 1 -- 172.21.15.5:0/2870187682 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c800125f0 con 0x7f0c98071f70 2024-03-18T15:34:44.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.240+0000 7f0c8f7fe640 1 -- 172.21.15.5:0/2870187682 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c80019070 con 0x7f0c98071f70 2024-03-18T15:34:44.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.240+0000 7f0c9d052640 1 -- 172.21.15.5:0/2870187682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c981e0ac0 con 0x7f0c98071f70 2024-03-18T15:34:44.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.240+0000 7f0c8f7fe640 1 -- 172.21.15.5:0/2870187682 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c800129b0 con 0x7f0c98071f70 2024-03-18T15:34:44.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.240+0000 7f0c9d052640 1 -- 172.21.15.5:0/2870187682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c98146d50 con 0x7f0c98071f70 2024-03-18T15:34:44.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.240+0000 7f0c9d052640 1 -- 172.21.15.5:0/2870187682 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0c9804f5c0 con 0x7f0c98071f70 2024-03-18T15:34:44.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.240+0000 7f0c8f7fe640 1 -- 172.21.15.5:0/2870187682 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f0c80012b50 con 0x7f0c98071f70 2024-03-18T15:34:44.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.240+0000 7f0c8f7fe640 1 --2- 172.21.15.5:0/2870187682 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0c78067690 0x7f0c78069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:44.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.240+0000 7f0c8f7fe640 1 -- 172.21.15.5:0/2870187682 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7f0c800862d0 con 0x7f0c98071f70 2024-03-18T15:34:44.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.240+0000 7f0c96d9d640 1 --2- 172.21.15.5:0/2870187682 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0c78067690 0x7f0c78069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:44.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.240+0000 7f0c8f7fe640 1 --2- 172.21.15.5:0/2870187682 >> [v2:172.21.15.5:6802/976264754,v1:172.21.15.5:6804/976264754] conn(0x7f0c7806d550 0x7f0c7806f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:44.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.240+0000 7f0c8f7fe640 1 -- 172.21.15.5:0/2870187682 --> [v2:172.21.15.5:6802/976264754,v1:172.21.15.5:6804/976264754] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0c8000cfb0 con 0x7f0c7806d550 2024-03-18T15:34:44.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.240+0000 7f0c8f7fe640 1 -- 172.21.15.5:0/2870187682 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f0c80086760 con 0x7f0c98071f70 2024-03-18T15:34:44.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.240+0000 7f0c9759e640 1 --2- 172.21.15.5:0/2870187682 >> [v2:172.21.15.5:6802/976264754,v1:172.21.15.5:6804/976264754] conn(0x7f0c7806d550 0x7f0c7806f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:44.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.244+0000 7f0c96d9d640 1 --2- 172.21.15.5:0/2870187682 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0c78067690 0x7f0c78069b50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0c880ae000 tx=0x7f0c880a7070 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:44.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.244+0000 7f0c9759e640 1 --2- 172.21.15.5:0/2870187682 >> [v2:172.21.15.5:6802/976264754,v1:172.21.15.5:6804/976264754] conn(0x7f0c7806d550 0x7f0c7806f970 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:44.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.244+0000 7f0c8f7fe640 1 -- 172.21.15.5:0/2870187682 <== osd.0 v2:172.21.15.5:6802/976264754 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0c8000cfb0 con 0x7f0c7806d550 2024-03-18T15:34:44.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.256+0000 7f0c9d052640 1 -- 172.21.15.5:0/2870187682 --> [v2:172.21.15.5:6802/976264754,v1:172.21.15.5:6804/976264754] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f0c98146a70 con 0x7f0c7806d550 2024-03-18T15:34:44.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.256+0000 7f0c8f7fe640 1 -- 172.21.15.5:0/2870187682 <== osd.0 v2:172.21.15.5:6802/976264754 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f0c98146a70 con 0x7f0c7806d550 2024-03-18T15:34:44.260 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-03-18T15:34:44.260 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-18T15:34:44.260 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-03-18T15:34:44.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.256+0000 7f0c9d052640 1 -- 172.21.15.5:0/2870187682 >> [v2:172.21.15.5:6802/976264754,v1:172.21.15.5:6804/976264754] conn(0x7f0c7806d550 msgr2=0x7f0c7806f970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:44.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.256+0000 7f0c9d052640 1 --2- 172.21.15.5:0/2870187682 >> [v2:172.21.15.5:6802/976264754,v1:172.21.15.5:6804/976264754] conn(0x7f0c7806d550 0x7f0c7806f970 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:44.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.256+0000 7f0c9d052640 1 -- 172.21.15.5:0/2870187682 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0c78067690 msgr2=0x7f0c78069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:44.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.256+0000 7f0c9d052640 1 --2- 172.21.15.5:0/2870187682 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0c78067690 0x7f0c78069b50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0c880ae000 tx=0x7f0c880a7070 comp rx=0 tx=0).stop 2024-03-18T15:34:44.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.256+0000 7f0c9d052640 1 -- 172.21.15.5:0/2870187682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c98071f70 msgr2=0x7f0c981e00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:44.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.256+0000 7f0c9d052640 1 --2- 172.21.15.5:0/2870187682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c98071f70 0x7f0c981e00f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0c8000d5f0 tx=0x7f0c8000dab0 comp rx=0 tx=0).stop 2024-03-18T15:34:44.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.256+0000 7f0c9759e640 1 -- 172.21.15.5:0/2870187682 reap_dead start 2024-03-18T15:34:44.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.256+0000 7f0c9d052640 1 -- 172.21.15.5:0/2870187682 shutdown_connections 2024-03-18T15:34:44.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.256+0000 7f0c9d052640 1 -- 172.21.15.5:0/2870187682 >> 172.21.15.5:0/2870187682 conn(0x7f0c980b5670 msgr2=0x7f0c980afbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:44.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.256+0000 7f0c9d052640 1 -- 172.21.15.5:0/2870187682 shutdown_connections 2024-03-18T15:34:44.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.256+0000 7f0c9d052640 1 -- 172.21.15.5:0/2870187682 wait complete. 2024-03-18T15:34:44.277 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-03-18T15:34:44.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.388+0000 7f04e19f5640 1 Processor -- start 2024-03-18T15:34:44.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.392+0000 7f04e19f5640 1 -- start start 2024-03-18T15:34:44.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.392+0000 7f04e19f5640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f04dc0b5710 0x7f04dc14a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:44.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.392+0000 7f04e19f5640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f04dc141600 0x7f04dc151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:44.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.392+0000 7f04e19f5640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04dc0b9d50 con 0x7f04dc14ae60 2024-03-18T15:34:44.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.392+0000 7f04e19f5640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f04dc141a90 con 0x7f04dc141600 2024-03-18T15:34:44.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.392+0000 7f04e19f5640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f04dc0b4320 con 0x7f04dc0b5710 2024-03-18T15:34:44.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.392+0000 7f04daffd640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f04dc0b5710 0x7f04dc14a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:44.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.392+0000 7f04db7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f04dc141600 0x7f04dc151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:44.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.392+0000 7f04da7fc640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f04dc14ae60 0x7f04dc140ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.5:35856/0 (socket says 172.21.15.5:35856) 2024-03-18T15:34:44.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.392+0000 7f04da7fc640 1 -- 172.21.15.5:0/3151316682 learned_addr learned my addr 172.21.15.5:0/3151316682 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:44.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.392+0000 7f04db7fe640 1 -- 172.21.15.5:0/3151316682 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f04dc0b5710 msgr2=0x7f04dc14a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:44.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.392+0000 7f04db7fe640 1 --2- 172.21.15.5:0/3151316682 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f04dc0b5710 0x7f04dc14a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:44.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.392+0000 7f04db7fe640 1 -- 172.21.15.5:0/3151316682 >> v1:172.21.15.80:6789/0 conn(0x7f04dc14ae60 legacy=0x7f04dc140ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:44.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.392+0000 7f04db7fe640 1 -- 172.21.15.5:0/3151316682 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04dc0b4c30 con 0x7f04dc141600 2024-03-18T15:34:44.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.392+0000 7f04daffd640 1 --2- 172.21.15.5:0/3151316682 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f04dc0b5710 0x7f04dc14a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:34:44.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.396+0000 7f04db7fe640 1 --2- 172.21.15.5:0/3151316682 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04dc141600 0x7f04dc151cb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f04d40097f0 tx=0x7f04d401c380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54793d037545cac2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:44.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.396+0000 7f04d9ffb640 1 -- 172.21.15.5:0/3151316682 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d40b0070 con 0x7f04dc141600 2024-03-18T15:34:44.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.396+0000 7f04d9ffb640 1 -- 172.21.15.5:0/3151316682 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04d40ab600 con 0x7f04dc141600 2024-03-18T15:34:44.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.396+0000 7f04d9ffb640 1 -- 172.21.15.5:0/3151316682 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d40ab9d0 con 0x7f04dc141600 2024-03-18T15:34:44.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.396+0000 7f04e19f5640 1 -- 172.21.15.5:0/3151316682 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04dc141600 msgr2=0x7f04dc151cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:44.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.396+0000 7f04e19f5640 1 --2- 172.21.15.5:0/3151316682 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04dc141600 0x7f04dc151cb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f04d40097f0 tx=0x7f04d401c380 comp rx=0 tx=0).stop 2024-03-18T15:34:44.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.396+0000 7f04e19f5640 1 -- 172.21.15.5:0/3151316682 shutdown_connections 2024-03-18T15:34:44.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.396+0000 7f04e19f5640 1 --2- 172.21.15.5:0/3151316682 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04dc141600 0x7f04dc151cb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:44.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.396+0000 7f04e19f5640 1 --2- 172.21.15.5:0/3151316682 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f04dc0b5710 0x7f04dc14a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:44.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.396+0000 7f04e19f5640 1 -- 172.21.15.5:0/3151316682 >> 172.21.15.5:0/3151316682 conn(0x7f04dc0673c0 msgr2=0x7f04dc0677d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:44.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.396+0000 7f04e19f5640 1 -- 172.21.15.5:0/3151316682 shutdown_connections 2024-03-18T15:34:44.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.396+0000 7f04e19f5640 1 -- 172.21.15.5:0/3151316682 wait complete. 2024-03-18T15:34:44.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.396+0000 7f04e19f5640 1 Processor -- start 2024-03-18T15:34:44.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.396+0000 7f04e19f5640 1 -- start start 2024-03-18T15:34:44.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.400+0000 7f04e19f5640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f04dc0b5710 0x7f04dc0bbd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:44.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.400+0000 7f04e19f5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04dc141600 0x7f04dc13b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:44.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.400+0000 7f04e19f5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04dc14ae60 0x7f04dc1e32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:44.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.400+0000 7f04e19f5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc0b50e0 con 0x7f04dc14ae60 2024-03-18T15:34:44.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.400+0000 7f04e19f5640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f04dc0b6940 con 0x7f04dc141600 2024-03-18T15:34:44.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.400+0000 7f04e19f5640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f04dc0b5260 con 0x7f04dc0b5710 2024-03-18T15:34:44.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.400+0000 7f04db7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04dc14ae60 0x7f04dc1e32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:44.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.400+0000 7f04db7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04dc14ae60 0x7f04dc1e32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34048/0 (socket says 172.21.15.5:34048) 2024-03-18T15:34:44.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.400+0000 7f04db7fe640 1 -- 172.21.15.5:0/660482518 learned_addr learned my addr 172.21.15.5:0/660482518 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:44.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.400+0000 7f04daffd640 1 --2- 172.21.15.5:0/660482518 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f04dc0b5710 0x7f04dc0bbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:44.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.400+0000 7f04da7fc640 1 --2- 172.21.15.5:0/660482518 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04dc141600 0x7f04dc13b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:44.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.400+0000 7f04db7fe640 1 -- 172.21.15.5:0/660482518 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f04dc0b5710 msgr2=0x7f04dc0bbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:44.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.400+0000 7f04db7fe640 1 --2- 172.21.15.5:0/660482518 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f04dc0b5710 0x7f04dc0bbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:44.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.400+0000 7f04db7fe640 1 -- 172.21.15.5:0/660482518 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04dc141600 msgr2=0x7f04dc13b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:44.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.400+0000 7f04db7fe640 1 --2- 172.21.15.5:0/660482518 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04dc141600 0x7f04dc13b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:44.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.400+0000 7f04db7fe640 1 -- 172.21.15.5:0/660482518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04dc147ca0 con 0x7f04dc14ae60 2024-03-18T15:34:44.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.400+0000 7f04da7fc640 1 --2- 172.21.15.5:0/660482518 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04dc141600 0x7f04dc13b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:34:44.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.400+0000 7f04db7fe640 1 --2- 172.21.15.5:0/660482518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04dc14ae60 0x7f04dc1e32c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f04d4019fb0 tx=0x7f04d4019120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:44.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.400+0000 7f04d3fff640 1 -- 172.21.15.5:0/660482518 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d40b0040 con 0x7f04dc14ae60 2024-03-18T15:34:44.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.400+0000 7f04d3fff640 1 -- 172.21.15.5:0/660482518 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04d40b9070 con 0x7f04dc14ae60 2024-03-18T15:34:44.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.400+0000 7f04d3fff640 1 -- 172.21.15.5:0/660482518 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d4006e20 con 0x7f04dc14ae60 2024-03-18T15:34:44.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.400+0000 7f04e19f5640 1 -- 172.21.15.5:0/660482518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04dc1e3800 con 0x7f04dc14ae60 2024-03-18T15:34:44.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.400+0000 7f04e19f5640 1 -- 172.21.15.5:0/660482518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04dc0bd8a0 con 0x7f04dc14ae60 2024-03-18T15:34:44.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.400+0000 7f04e19f5640 1 -- 172.21.15.5:0/660482518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04dc04f550 con 0x7f04dc14ae60 2024-03-18T15:34:44.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.400+0000 7f04d3fff640 1 -- 172.21.15.5:0/660482518 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f04d40b4650 con 0x7f04dc14ae60 2024-03-18T15:34:44.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.404+0000 7f04d3fff640 1 --2- 172.21.15.5:0/660482518 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f04bc067660 0x7f04bc069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:44.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.404+0000 7f04d3fff640 1 -- 172.21.15.5:0/660482518 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7f04d411f180 con 0x7f04dc14ae60 2024-03-18T15:34:44.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.404+0000 7f04daffd640 1 --2- 172.21.15.5:0/660482518 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f04bc067660 0x7f04bc069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:44.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.404+0000 7f04daffd640 1 --2- 172.21.15.5:0/660482518 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f04bc067660 0x7f04bc069b20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f04cc009bd0 tx=0x7f04cc00b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:44.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.408+0000 7f04d3fff640 1 -- 172.21.15.5:0/660482518 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f04d4016090 con 0x7f04dc14ae60 2024-03-18T15:34:44.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.568+0000 7f04e19f5640 1 -- 172.21.15.5:0/660482518 --> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f04dc0b7140 con 0x7f04bc067660 2024-03-18T15:34:44.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.568+0000 7f04d3fff640 1 -- 172.21.15.5:0/660482518 <== mgr.4110 v2:172.21.15.5:6832/3362034860 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f04dc0b7140 con 0x7f04bc067660 2024-03-18T15:34:44.570 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 0 to deep-scrub 2024-03-18T15:34:44.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.568+0000 7f04e19f5640 1 -- 172.21.15.5:0/660482518 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f04bc067660 msgr2=0x7f04bc069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:44.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.568+0000 7f04e19f5640 1 --2- 172.21.15.5:0/660482518 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f04bc067660 0x7f04bc069b20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f04cc009bd0 tx=0x7f04cc00b040 comp rx=0 tx=0).stop 2024-03-18T15:34:44.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.568+0000 7f04e19f5640 1 -- 172.21.15.5:0/660482518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04dc14ae60 msgr2=0x7f04dc1e32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:44.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.568+0000 7f04e19f5640 1 --2- 172.21.15.5:0/660482518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04dc14ae60 0x7f04dc1e32c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f04d4019fb0 tx=0x7f04d4019120 comp rx=0 tx=0).stop 2024-03-18T15:34:44.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.568+0000 7f04e19f5640 1 -- 172.21.15.5:0/660482518 shutdown_connections 2024-03-18T15:34:44.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.568+0000 7f04e19f5640 1 --2- 172.21.15.5:0/660482518 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f04bc067660 0x7f04bc069b20 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:44.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.568+0000 7f04e19f5640 1 --2- 172.21.15.5:0/660482518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f04dc14ae60 0x7f04dc1e32c0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:44.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.568+0000 7f04e19f5640 1 --2- 172.21.15.5:0/660482518 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04dc141600 0x7f04dc13b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:44.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.568+0000 7f04e19f5640 1 --2- 172.21.15.5:0/660482518 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f04dc0b5710 0x7f04dc0bbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:44.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.568+0000 7f04e19f5640 1 -- 172.21.15.5:0/660482518 >> 172.21.15.5:0/660482518 conn(0x7f04dc0673c0 msgr2=0x7f04dc148da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:44.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.568+0000 7f04e19f5640 1 -- 172.21.15.5:0/660482518 shutdown_connections 2024-03-18T15:34:44.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.568+0000 7f04e19f5640 1 -- 172.21.15.5:0/660482518 wait complete. 2024-03-18T15:34:44.589 INFO:tasks.ceph:Scrubbing osd.1 2024-03-18T15:34:44.590 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-03-18T15:34:44.635 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-18T15:34:44.637 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-18T15:34:44.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.712+0000 7f96ba2c0640 1 Processor -- start 2024-03-18T15:34:44.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.712+0000 7f96ba2c0640 1 -- start start 2024-03-18T15:34:44.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.712+0000 7f96ba2c0640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f96b406d300 0x7f96b4145250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:44.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.712+0000 7f96ba2c0640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96b40afd60 con 0x7f96b40b4820 2024-03-18T15:34:44.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.712+0000 7f96ba2c0640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96b40b52b0 con 0x7f96b406f640 2024-03-18T15:34:44.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.712+0000 7f96ba2c0640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f96b406d6e0 con 0x7f96b406d300 2024-03-18T15:34:44.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.712+0000 7f96b37fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f96b40b4820 0x7f96b413efb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:39850/0 (socket says 172.21.15.5:39850) 2024-03-18T15:34:44.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.712+0000 7f96b37fe640 1 -- 172.21.15.5:0/3365387846 learned_addr learned my addr 172.21.15.5:0/3365387846 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:44.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.712+0000 7f96b2ffd640 1 -- 172.21.15.5:0/3365387846 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3232601794 0 0) 0x7f96b40afd60 con 0x7f96b40b4820 2024-03-18T15:34:44.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.712+0000 7f96b3fff640 1 --2- 172.21.15.5:0/3365387846 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f96b406d300 0x7f96b4145250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:44.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.712+0000 7f96b2ffd640 1 -- 172.21.15.5:0/3365387846 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f969c003530 con 0x7f96b40b4820 2024-03-18T15:34:44.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.712+0000 7f96b2ffd640 1 -- 172.21.15.5:0/3365387846 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2619523760 0 0) 0x7f969c003530 con 0x7f96b40b4820 2024-03-18T15:34:44.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.712+0000 7f96b2ffd640 1 -- 172.21.15.5:0/3365387846 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f96b406d300 msgr2=0x7f96b4145250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:44.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.712+0000 7f96b2ffd640 1 --2- 172.21.15.5:0/3365387846 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f96b406d300 0x7f96b4145250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:44.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.716+0000 7f96b2ffd640 1 -- 172.21.15.5:0/3365387846 >> v1:172.21.15.80:6789/0 conn(0x7f96b406f640 legacy=0x7f96b414fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:44.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.716+0000 7f96b2ffd640 1 -- 172.21.15.5:0/3365387846 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b4070960 con 0x7f96b40b4820 2024-03-18T15:34:44.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.716+0000 7f96b2ffd640 1 -- 172.21.15.5:0/3365387846 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3837549003 0 0) 0x7f96a4001680 con 0x7f96b40b4820 2024-03-18T15:34:44.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.716+0000 7f96b2ffd640 1 -- 172.21.15.5:0/3365387846 >> v1:172.21.15.5:6789/0 conn(0x7f96b40b4820 legacy=0x7f96b413efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:44.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.716+0000 7f96b2ffd640 1 --2- 172.21.15.5:0/3365387846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f969c003ad0 0x7f969c003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:44.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.716+0000 7f96b2ffd640 1 --2- 172.21.15.5:0/3365387846 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f96b406d300 0x7f969c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:44.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.716+0000 7f96b2ffd640 1 --2- 172.21.15.5:0/3365387846 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f969c004f50 0x7f969c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:44.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.716+0000 7f96b2ffd640 1 -- 172.21.15.5:0/3365387846 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f96b406d6e0 con 0x7f969c003ad0 2024-03-18T15:34:44.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.716+0000 7f96b2ffd640 1 -- 172.21.15.5:0/3365387846 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f969c00bf10 con 0x7f969c004f50 2024-03-18T15:34:44.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.716+0000 7f96b2ffd640 1 -- 172.21.15.5:0/3365387846 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f969c00c090 con 0x7f96b406d300 2024-03-18T15:34:44.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.716+0000 7f96b3fff640 1 --2- 172.21.15.5:0/3365387846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f969c003ad0 0x7f969c003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:44.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.716+0000 7f96b885d640 1 --2- 172.21.15.5:0/3365387846 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f969c004f50 0x7f969c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:44.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.716+0000 7f96b37fe640 1 --2- 172.21.15.5:0/3365387846 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f96b406d300 0x7f969c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:44.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.716+0000 7f96b3fff640 1 -- 172.21.15.5:0/3365387846 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f96b406d300 msgr2=0x7f969c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:44.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.716+0000 7f96b3fff640 1 --2- 172.21.15.5:0/3365387846 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f96b406d300 0x7f969c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:44.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.716+0000 7f96b3fff640 1 -- 172.21.15.5:0/3365387846 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f969c004f50 msgr2=0x7f969c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:44.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.716+0000 7f96b3fff640 1 --2- 172.21.15.5:0/3365387846 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f969c004f50 0x7f969c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:44.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.716+0000 7f96b3fff640 1 -- 172.21.15.5:0/3365387846 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f969c003530 con 0x7f969c003ad0 2024-03-18T15:34:44.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.716+0000 7f96b3fff640 1 --2- 172.21.15.5:0/3365387846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f969c003ad0 0x7f969c003ef0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f96a800ba60 tx=0x7f96a801c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f30d27a4fd1ab024 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:44.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.716+0000 7f96b2ffd640 1 -- 172.21.15.5:0/3365387846 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a800b600 con 0x7f969c003ad0 2024-03-18T15:34:44.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.716+0000 7f96b2ffd640 1 -- 172.21.15.5:0/3365387846 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96a80b0070 con 0x7f969c003ad0 2024-03-18T15:34:44.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.716+0000 7f96b2ffd640 1 -- 172.21.15.5:0/3365387846 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a80ab880 con 0x7f969c003ad0 2024-03-18T15:34:44.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.720+0000 7f96ba2c0640 1 -- 172.21.15.5:0/3365387846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f969c003ad0 msgr2=0x7f969c003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:44.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.720+0000 7f96ba2c0640 1 --2- 172.21.15.5:0/3365387846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f969c003ad0 0x7f969c003ef0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f96a800ba60 tx=0x7f96a801c650 comp rx=0 tx=0).stop 2024-03-18T15:34:44.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.720+0000 7f96b885d640 1 -- 172.21.15.5:0/3365387846 reap_dead start 2024-03-18T15:34:44.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.720+0000 7f96ba2c0640 1 -- 172.21.15.5:0/3365387846 shutdown_connections 2024-03-18T15:34:44.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.720+0000 7f96ba2c0640 1 -- 172.21.15.5:0/3365387846 >> 172.21.15.5:0/3365387846 conn(0x7f96b40b5670 msgr2=0x7f96b40b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:44.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.720+0000 7f96ba2c0640 1 -- 172.21.15.5:0/3365387846 shutdown_connections 2024-03-18T15:34:44.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.720+0000 7f96ba2c0640 1 -- 172.21.15.5:0/3365387846 wait complete. 2024-03-18T15:34:44.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.720+0000 7f96ba2c0640 1 Processor -- start 2024-03-18T15:34:44.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.720+0000 7f96ba2c0640 1 -- start start 2024-03-18T15:34:44.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.720+0000 7f96ba2c0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96b4146610 0x7f96b4147950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:44.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.720+0000 7f96ba2c0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f96b41e1b10 0x7f96b41e1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:44.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.720+0000 7f96ba2c0640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f96b41e24d0 0x7f96b4138af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:44.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.720+0000 7f96ba2c0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f96b41457e0 con 0x7f96b4146610 2024-03-18T15:34:44.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.720+0000 7f96ba2c0640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f96b4145660 con 0x7f96b41e1b10 2024-03-18T15:34:44.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.720+0000 7f96ba2c0640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f96b4151a60 con 0x7f96b41e24d0 2024-03-18T15:34:44.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.720+0000 7f96b3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96b4146610 0x7f96b4147950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:44.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.720+0000 7f96b3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96b4146610 0x7f96b4147950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34070/0 (socket says 172.21.15.5:34070) 2024-03-18T15:34:44.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.720+0000 7f96b3fff640 1 -- 172.21.15.5:0/3480673715 learned_addr learned my addr 172.21.15.5:0/3480673715 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:44.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.720+0000 7f96b37fe640 1 --2- 172.21.15.5:0/3480673715 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f96b41e1b10 0x7f96b41e1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:44.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.720+0000 7f96b885d640 1 --2- 172.21.15.5:0/3480673715 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f96b41e24d0 0x7f96b4138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:44.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.720+0000 7f96b3fff640 1 -- 172.21.15.5:0/3480673715 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f96b41e24d0 msgr2=0x7f96b4138af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:44.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.720+0000 7f96b3fff640 1 --2- 172.21.15.5:0/3480673715 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f96b41e24d0 0x7f96b4138af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:44.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.720+0000 7f96b3fff640 1 -- 172.21.15.5:0/3480673715 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f96b41e1b10 msgr2=0x7f96b41e1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:44.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.720+0000 7f96b3fff640 1 --2- 172.21.15.5:0/3480673715 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f96b41e1b10 0x7f96b41e1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:44.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.720+0000 7f96b3fff640 1 -- 172.21.15.5:0/3480673715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b41e2950 con 0x7f96b4146610 2024-03-18T15:34:44.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.720+0000 7f96b3fff640 1 --2- 172.21.15.5:0/3480673715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96b4146610 0x7f96b4147950 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f96a80ad040 tx=0x7f969c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:44.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.720+0000 7f96b0ff9640 1 -- 172.21.15.5:0/3480673715 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a8004220 con 0x7f96b4146610 2024-03-18T15:34:44.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.720+0000 7f96b0ff9640 1 -- 172.21.15.5:0/3480673715 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96a80b8070 con 0x7f96b4146610 2024-03-18T15:34:44.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.720+0000 7f96ba2c0640 1 -- 172.21.15.5:0/3480673715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96b4149190 con 0x7f96b4146610 2024-03-18T15:34:44.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.724+0000 7f96b0ff9640 1 -- 172.21.15.5:0/3480673715 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a8003620 con 0x7f96b4146610 2024-03-18T15:34:44.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.724+0000 7f96ba2c0640 1 -- 172.21.15.5:0/3480673715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96b41508b0 con 0x7f96b4146610 2024-03-18T15:34:44.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.724+0000 7f96ba2c0640 1 -- 172.21.15.5:0/3480673715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f96b4148c60 con 0x7f96b4146610 2024-03-18T15:34:44.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.724+0000 7f96b0ff9640 1 -- 172.21.15.5:0/3480673715 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f96a80037c0 con 0x7f96b4146610 2024-03-18T15:34:44.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.724+0000 7f96b0ff9640 1 --2- 172.21.15.5:0/3480673715 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f96a0067660 0x7f96a0069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:44.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.724+0000 7f96b0ff9640 1 -- 172.21.15.5:0/3480673715 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7f96a811ff80 con 0x7f96b4146610 2024-03-18T15:34:44.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.724+0000 7f96b37fe640 1 --2- 172.21.15.5:0/3480673715 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f96a0067660 0x7f96a0069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:44.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.724+0000 7f96b0ff9640 1 --2- 172.21.15.5:0/3480673715 >> [v2:172.21.15.5:6800/3901761126,v1:172.21.15.5:6801/3901761126] conn(0x7f96a006d520 0x7f96a006f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:44.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.724+0000 7f96b0ff9640 1 -- 172.21.15.5:0/3480673715 --> [v2:172.21.15.5:6800/3901761126,v1:172.21.15.5:6801/3901761126] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f96a80baec0 con 0x7f96a006d520 2024-03-18T15:34:44.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.724+0000 7f96b0ff9640 1 -- 172.21.15.5:0/3480673715 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f96a8120410 con 0x7f96b4146610 2024-03-18T15:34:44.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.724+0000 7f96b885d640 1 --2- 172.21.15.5:0/3480673715 >> [v2:172.21.15.5:6800/3901761126,v1:172.21.15.5:6801/3901761126] conn(0x7f96a006d520 0x7f96a006f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:44.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.724+0000 7f96b37fe640 1 --2- 172.21.15.5:0/3480673715 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f96a0067660 0x7f96a0069b20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f96b4141b80 tx=0x7f96a4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:44.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.724+0000 7f96b885d640 1 --2- 172.21.15.5:0/3480673715 >> [v2:172.21.15.5:6800/3901761126,v1:172.21.15.5:6801/3901761126] conn(0x7f96a006d520 0x7f96a006f940 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:44.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.724+0000 7f96b0ff9640 1 -- 172.21.15.5:0/3480673715 <== osd.1 v2:172.21.15.5:6800/3901761126 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f96a80baec0 con 0x7f96a006d520 2024-03-18T15:34:44.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.740+0000 7f96ba2c0640 1 -- 172.21.15.5:0/3480673715 --> [v2:172.21.15.5:6800/3901761126,v1:172.21.15.5:6801/3901761126] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f96b41474a0 con 0x7f96a006d520 2024-03-18T15:34:44.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.740+0000 7f96b0ff9640 1 -- 172.21.15.5:0/3480673715 <== osd.1 v2:172.21.15.5:6800/3901761126 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f96b41474a0 con 0x7f96a006d520 2024-03-18T15:34:44.743 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-03-18T15:34:44.743 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-18T15:34:44.743 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-03-18T15:34:44.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.740+0000 7f96ba2c0640 1 -- 172.21.15.5:0/3480673715 >> [v2:172.21.15.5:6800/3901761126,v1:172.21.15.5:6801/3901761126] conn(0x7f96a006d520 msgr2=0x7f96a006f940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:44.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.740+0000 7f96ba2c0640 1 --2- 172.21.15.5:0/3480673715 >> [v2:172.21.15.5:6800/3901761126,v1:172.21.15.5:6801/3901761126] conn(0x7f96a006d520 0x7f96a006f940 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:44.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.740+0000 7f96ba2c0640 1 -- 172.21.15.5:0/3480673715 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f96a0067660 msgr2=0x7f96a0069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:44.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.740+0000 7f96ba2c0640 1 --2- 172.21.15.5:0/3480673715 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f96a0067660 0x7f96a0069b20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f96b4141b80 tx=0x7f96a4009040 comp rx=0 tx=0).stop 2024-03-18T15:34:44.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.740+0000 7f96ba2c0640 1 -- 172.21.15.5:0/3480673715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96b4146610 msgr2=0x7f96b4147950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:44.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.740+0000 7f96ba2c0640 1 --2- 172.21.15.5:0/3480673715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96b4146610 0x7f96b4147950 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f96a80ad040 tx=0x7f969c005840 comp rx=0 tx=0).stop 2024-03-18T15:34:44.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.740+0000 7f96b885d640 1 -- 172.21.15.5:0/3480673715 reap_dead start 2024-03-18T15:34:44.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.740+0000 7f96ba2c0640 1 -- 172.21.15.5:0/3480673715 shutdown_connections 2024-03-18T15:34:44.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.740+0000 7f96ba2c0640 1 -- 172.21.15.5:0/3480673715 >> 172.21.15.5:0/3480673715 conn(0x7f96b40b5670 msgr2=0x7f96b406c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:44.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.740+0000 7f96ba2c0640 1 -- 172.21.15.5:0/3480673715 shutdown_connections 2024-03-18T15:34:44.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.740+0000 7f96ba2c0640 1 -- 172.21.15.5:0/3480673715 wait complete. 2024-03-18T15:34:44.759 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-03-18T15:34:44.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.920+0000 7f84f418a640 1 Processor -- start 2024-03-18T15:34:44.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.920+0000 7f84f418a640 1 -- start start 2024-03-18T15:34:44.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.920+0000 7f84f418a640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f84ec070320 0x7f84ec147530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:44.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.920+0000 7f84f418a640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84ec06b8c0 con 0x7f84ec06b4e0 2024-03-18T15:34:44.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.920+0000 7f84f418a640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84ec141b00 con 0x7f84ec147a70 2024-03-18T15:34:44.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.920+0000 7f84f418a640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f84ec06ee50 con 0x7f84ec070320 2024-03-18T15:34:44.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.920+0000 7f84f1725640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f84ec070320 0x7f84ec147530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:44.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.920+0000 7f84f1f26640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f84ec06b4e0 0x7f84ec1413f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:39858/0 (socket says 172.21.15.5:39858) 2024-03-18T15:34:44.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.920+0000 7f84f1f26640 1 -- 172.21.15.5:0/686485733 learned_addr learned my addr 172.21.15.5:0/686485733 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:44.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.920+0000 7f84f1725640 1 -- 172.21.15.5:0/686485733 >> v1:172.21.15.80:6789/0 conn(0x7f84ec147a70 legacy=0x7f84ec151cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:44.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.920+0000 7f84f1725640 1 -- 172.21.15.5:0/686485733 >> v1:172.21.15.5:6789/0 conn(0x7f84ec06b4e0 legacy=0x7f84ec1413f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:44.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.920+0000 7f84f1725640 1 -- 172.21.15.5:0/686485733 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84ec0707b0 con 0x7f84ec070320 2024-03-18T15:34:44.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.924+0000 7f84f1725640 1 --2- 172.21.15.5:0/686485733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84ec070320 0x7f84ec147530 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f84dc0097c0 tx=0x7f84dc01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f26cd08bc22ef26 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:44.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.924+0000 7f84f0f24640 1 -- 172.21.15.5:0/686485733 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84dc0b0070 con 0x7f84ec070320 2024-03-18T15:34:44.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.924+0000 7f84f0f24640 1 -- 172.21.15.5:0/686485733 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84dc0ab600 con 0x7f84ec070320 2024-03-18T15:34:44.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.924+0000 7f84f0f24640 1 -- 172.21.15.5:0/686485733 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84dc0a4e60 con 0x7f84ec070320 2024-03-18T15:34:44.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.924+0000 7f84f418a640 1 -- 172.21.15.5:0/686485733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84ec070320 msgr2=0x7f84ec147530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:44.927 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.924+0000 7f84f418a640 1 --2- 172.21.15.5:0/686485733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84ec070320 0x7f84ec147530 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f84dc0097c0 tx=0x7f84dc01c3d0 comp rx=0 tx=0).stop 2024-03-18T15:34:44.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.924+0000 7f84f418a640 1 -- 172.21.15.5:0/686485733 shutdown_connections 2024-03-18T15:34:44.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.924+0000 7f84f418a640 1 --2- 172.21.15.5:0/686485733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84ec070320 0x7f84ec147530 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:44.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.924+0000 7f84f418a640 1 -- 172.21.15.5:0/686485733 >> 172.21.15.5:0/686485733 conn(0x7f84ec0b7a80 msgr2=0x7f84ec0b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:44.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.924+0000 7f84f418a640 1 -- 172.21.15.5:0/686485733 shutdown_connections 2024-03-18T15:34:44.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.924+0000 7f84f418a640 1 -- 172.21.15.5:0/686485733 wait complete. 2024-03-18T15:34:44.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.924+0000 7f84f418a640 1 Processor -- start 2024-03-18T15:34:44.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.924+0000 7f84f418a640 1 -- start start 2024-03-18T15:34:44.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.924+0000 7f84f418a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84ec06b4e0 0x7f84ec0adab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:44.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.924+0000 7f84f418a640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84ec147a70 0x7f84ec0ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:44.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.924+0000 7f84f418a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84ec0af980 0x7f84ec13acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:44.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.924+0000 7f84f418a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f84ec06e850 con 0x7f84ec06b4e0 2024-03-18T15:34:44.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.924+0000 7f84f418a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f84ec06e6d0 con 0x7f84ec0af980 2024-03-18T15:34:44.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.924+0000 7f84f418a640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f84ec155c00 con 0x7f84ec147a70 2024-03-18T15:34:44.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.924+0000 7f84f1f26640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84ec06b4e0 0x7f84ec0adab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:44.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.924+0000 7f84f1f26640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84ec06b4e0 0x7f84ec0adab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34088/0 (socket says 172.21.15.5:34088) 2024-03-18T15:34:44.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.924+0000 7f84f1f26640 1 -- 172.21.15.5:0/1476893116 learned_addr learned my addr 172.21.15.5:0/1476893116 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:44.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.928+0000 7f84f1725640 1 --2- 172.21.15.5:0/1476893116 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84ec147a70 0x7f84ec0ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:44.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.928+0000 7f84f1f26640 1 -- 172.21.15.5:0/1476893116 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84ec147a70 msgr2=0x7f84ec0ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:44.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.928+0000 7f84f2727640 1 --2- 172.21.15.5:0/1476893116 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84ec0af980 0x7f84ec13acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:44.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.928+0000 7f84f1f26640 1 --2- 172.21.15.5:0/1476893116 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84ec147a70 0x7f84ec0ae8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:44.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.928+0000 7f84f1f26640 1 -- 172.21.15.5:0/1476893116 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84ec0af980 msgr2=0x7f84ec13acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:44.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.928+0000 7f84f1f26640 1 --2- 172.21.15.5:0/1476893116 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84ec0af980 0x7f84ec13acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:44.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.928+0000 7f84f1f26640 1 -- 172.21.15.5:0/1476893116 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84ec0b0b50 con 0x7f84ec06b4e0 2024-03-18T15:34:44.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.928+0000 7f84f1f26640 1 --2- 172.21.15.5:0/1476893116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84ec06b4e0 0x7f84ec0adab0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f84e800c3a0 tx=0x7f84e800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:44.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.928+0000 7f84e27fc640 1 -- 172.21.15.5:0/1476893116 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e8017070 con 0x7f84ec06b4e0 2024-03-18T15:34:44.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.928+0000 7f84e27fc640 1 -- 172.21.15.5:0/1476893116 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84e8005cf0 con 0x7f84ec06b4e0 2024-03-18T15:34:44.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.928+0000 7f84f418a640 1 -- 172.21.15.5:0/1476893116 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84ec142150 con 0x7f84ec06b4e0 2024-03-18T15:34:44.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.928+0000 7f84e27fc640 1 -- 172.21.15.5:0/1476893116 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e80072f0 con 0x7f84ec06b4e0 2024-03-18T15:34:44.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.928+0000 7f84f418a640 1 -- 172.21.15.5:0/1476893116 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84ec0afe60 con 0x7f84ec06b4e0 2024-03-18T15:34:44.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.928+0000 7f84f418a640 1 -- 172.21.15.5:0/1476893116 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84ec04f550 con 0x7f84ec06b4e0 2024-03-18T15:34:44.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.928+0000 7f84e27fc640 1 -- 172.21.15.5:0/1476893116 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f84e801a070 con 0x7f84ec06b4e0 2024-03-18T15:34:44.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.928+0000 7f84e27fc640 1 --2- 172.21.15.5:0/1476893116 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f84d0067690 0x7f84d0069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:44.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.928+0000 7f84f1725640 1 --2- 172.21.15.5:0/1476893116 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f84d0067690 0x7f84d0069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:44.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.928+0000 7f84e27fc640 1 -- 172.21.15.5:0/1476893116 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7f84e80848f0 con 0x7f84ec06b4e0 2024-03-18T15:34:44.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.928+0000 7f84f1725640 1 --2- 172.21.15.5:0/1476893116 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f84d0067690 0x7f84d0069b50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f84dc0ad000 tx=0x7f84dc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:44.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:44.932+0000 7f84e27fc640 1 -- 172.21.15.5:0/1476893116 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f84e8055900 con 0x7f84ec06b4e0 2024-03-18T15:34:45.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.092+0000 7f84f418a640 1 -- 172.21.15.5:0/1476893116 --> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f84ec0b4990 con 0x7f84d0067690 2024-03-18T15:34:45.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.092+0000 7f84e27fc640 1 -- 172.21.15.5:0/1476893116 <== mgr.4110 v2:172.21.15.5:6832/3362034860 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f84ec0b4990 con 0x7f84d0067690 2024-03-18T15:34:45.097 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 1 to deep-scrub 2024-03-18T15:34:45.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.096+0000 7f84f418a640 1 -- 172.21.15.5:0/1476893116 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f84d0067690 msgr2=0x7f84d0069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:45.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.096+0000 7f84f418a640 1 --2- 172.21.15.5:0/1476893116 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f84d0067690 0x7f84d0069b50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f84dc0ad000 tx=0x7f84dc008040 comp rx=0 tx=0).stop 2024-03-18T15:34:45.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.096+0000 7f84f418a640 1 -- 172.21.15.5:0/1476893116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84ec06b4e0 msgr2=0x7f84ec0adab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:45.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.096+0000 7f84f418a640 1 --2- 172.21.15.5:0/1476893116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84ec06b4e0 0x7f84ec0adab0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f84e800c3a0 tx=0x7f84e800c860 comp rx=0 tx=0).stop 2024-03-18T15:34:45.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.096+0000 7f84f418a640 1 -- 172.21.15.5:0/1476893116 shutdown_connections 2024-03-18T15:34:45.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.096+0000 7f84f418a640 1 --2- 172.21.15.5:0/1476893116 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f84d0067690 0x7f84d0069b50 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:45.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.096+0000 7f84f418a640 1 --2- 172.21.15.5:0/1476893116 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84ec0af980 0x7f84ec13acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:45.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.096+0000 7f84f418a640 1 --2- 172.21.15.5:0/1476893116 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84ec147a70 0x7f84ec0ae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:45.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.096+0000 7f84f418a640 1 --2- 172.21.15.5:0/1476893116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84ec06b4e0 0x7f84ec0adab0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:45.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.096+0000 7f84f418a640 1 -- 172.21.15.5:0/1476893116 >> 172.21.15.5:0/1476893116 conn(0x7f84ec0b7a80 msgr2=0x7f84ec0b87d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:45.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.096+0000 7f84f418a640 1 -- 172.21.15.5:0/1476893116 shutdown_connections 2024-03-18T15:34:45.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.096+0000 7f84f418a640 1 -- 172.21.15.5:0/1476893116 wait complete. 2024-03-18T15:34:45.116 INFO:tasks.ceph:Scrubbing osd.2 2024-03-18T15:34:45.116 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-03-18T15:34:45.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.240+0000 7fd3bd486640 1 Processor -- start 2024-03-18T15:34:45.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.240+0000 7fd3bd486640 1 -- start start 2024-03-18T15:34:45.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.240+0000 7fd3bd486640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd3b806d300 0x7fd3b80aea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:45.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.240+0000 7fd3bd486640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd3b814b230 0x7fd3b8072490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:45.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.240+0000 7fd3bd486640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3b806d6e0 con 0x7fd3b80ad8e0 2024-03-18T15:34:45.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.240+0000 7fd3bd486640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fd3b80b5370 con 0x7fd3b806d300 2024-03-18T15:34:45.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.240+0000 7fd3bd486640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fd3b80ae150 con 0x7fd3b814b230 2024-03-18T15:34:45.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.240+0000 7fd3b6ffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fd3b806d300 0x7fd3b80aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:45.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.240+0000 7fd3b67fc640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7fd3b80ad8e0 0x7fd3b814ab20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:39864/0 (socket says 172.21.15.5:39864) 2024-03-18T15:34:45.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.240+0000 7fd3b67fc640 1 -- 172.21.15.5:0/1220501118 learned_addr learned my addr 172.21.15.5:0/1220501118 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:45.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.240+0000 7fd3b77fe640 1 --2- 172.21.15.5:0/1220501118 >> v2:172.21.15.80:3300/0 conn(0x7fd3b814b230 0x7fd3b8072490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:45.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.240+0000 7fd3b6ffd640 1 -- 172.21.15.5:0/1220501118 >> v2:172.21.15.80:3300/0 conn(0x7fd3b814b230 msgr2=0x7fd3b8072490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:45.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.240+0000 7fd3b6ffd640 1 --2- 172.21.15.5:0/1220501118 >> v2:172.21.15.80:3300/0 conn(0x7fd3b814b230 0x7fd3b8072490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:45.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.240+0000 7fd3b6ffd640 1 -- 172.21.15.5:0/1220501118 >> v1:172.21.15.5:6789/0 conn(0x7fd3b80ad8e0 legacy=0x7fd3b814ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:45.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.240+0000 7fd3b6ffd640 1 -- 172.21.15.5:0/1220501118 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3b80b4d20 con 0x7fd3b806d300 2024-03-18T15:34:45.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.240+0000 7fd3b5ffb640 1 -- 172.21.15.5:0/1220501118 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 677390797 0 0) 0x7fd3b806d6e0 con 0x7fd3b80ad8e0 2024-03-18T15:34:45.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.244+0000 7fd3b6ffd640 1 --2- 172.21.15.5:0/1220501118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3b806d300 0x7fd3b80aea60 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd3b80b4c10 tx=0x7fd3a801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f836052ae446502b server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:45.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.244+0000 7fd3b5ffb640 1 -- 172.21.15.5:0/1220501118 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a80b0070 con 0x7fd3b806d300 2024-03-18T15:34:45.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.244+0000 7fd3b5ffb640 1 -- 172.21.15.5:0/1220501118 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3a80ab600 con 0x7fd3b806d300 2024-03-18T15:34:45.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.244+0000 7fd3b5ffb640 1 -- 172.21.15.5:0/1220501118 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a80ab9d0 con 0x7fd3b806d300 2024-03-18T15:34:45.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.244+0000 7fd3bd486640 1 -- 172.21.15.5:0/1220501118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3b806d300 msgr2=0x7fd3b80aea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:45.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.244+0000 7fd3bd486640 1 --2- 172.21.15.5:0/1220501118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3b806d300 0x7fd3b80aea60 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd3b80b4c10 tx=0x7fd3a801c3f0 comp rx=0 tx=0).stop 2024-03-18T15:34:45.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.244+0000 7fd3bd486640 1 -- 172.21.15.5:0/1220501118 shutdown_connections 2024-03-18T15:34:45.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.244+0000 7fd3bd486640 1 --2- 172.21.15.5:0/1220501118 >> v2:172.21.15.80:3300/0 conn(0x7fd3b814b230 0x7fd3b8072490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:45.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.244+0000 7fd3bd486640 1 --2- 172.21.15.5:0/1220501118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3b806d300 0x7fd3b80aea60 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:45.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.244+0000 7fd3bd486640 1 -- 172.21.15.5:0/1220501118 >> 172.21.15.5:0/1220501118 conn(0x7fd3b80b5670 msgr2=0x7fd3b80b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:45.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.244+0000 7fd3bd486640 1 -- 172.21.15.5:0/1220501118 shutdown_connections 2024-03-18T15:34:45.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.244+0000 7fd3bd486640 1 -- 172.21.15.5:0/1220501118 wait complete. 2024-03-18T15:34:45.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.244+0000 7fd3bd486640 1 Processor -- start 2024-03-18T15:34:45.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.244+0000 7fd3bd486640 1 -- start start 2024-03-18T15:34:45.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.244+0000 7fd3bd486640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd3b806d300 0x7fd3b80b8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:45.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.244+0000 7fd3bd486640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3b80ad8e0 0x7fd3b81e39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:45.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.244+0000 7fd3bd486640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd3b814b230 0x7fd3b81e3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:45.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.244+0000 7fd3bd486640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b81548f0 con 0x7fd3b80ad8e0 2024-03-18T15:34:45.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.244+0000 7fd3bd486640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b8154770 con 0x7fd3b814b230 2024-03-18T15:34:45.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.244+0000 7fd3bd486640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fd3b8154a70 con 0x7fd3b806d300 2024-03-18T15:34:45.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.244+0000 7fd3b67fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3b80ad8e0 0x7fd3b81e39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:45.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.244+0000 7fd3b67fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3b80ad8e0 0x7fd3b81e39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34108/0 (socket says 172.21.15.5:34108) 2024-03-18T15:34:45.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.244+0000 7fd3b67fc640 1 -- 172.21.15.5:0/2612523411 learned_addr learned my addr 172.21.15.5:0/2612523411 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:45.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.244+0000 7fd3b77fe640 1 --2- 172.21.15.5:0/2612523411 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd3b814b230 0x7fd3b81e3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:45.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.244+0000 7fd3b6ffd640 1 --2- 172.21.15.5:0/2612523411 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd3b806d300 0x7fd3b80b8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:45.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.244+0000 7fd3b67fc640 1 -- 172.21.15.5:0/2612523411 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd3b806d300 msgr2=0x7fd3b80b8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:45.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.244+0000 7fd3b67fc640 1 --2- 172.21.15.5:0/2612523411 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd3b806d300 0x7fd3b80b8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:45.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.244+0000 7fd3b67fc640 1 -- 172.21.15.5:0/2612523411 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd3b814b230 msgr2=0x7fd3b81e3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:45.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.244+0000 7fd3b67fc640 1 --2- 172.21.15.5:0/2612523411 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd3b814b230 0x7fd3b81e3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:45.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.244+0000 7fd3b67fc640 1 -- 172.21.15.5:0/2612523411 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3b80b2470 con 0x7fd3b80ad8e0 2024-03-18T15:34:45.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.248+0000 7fd3b6ffd640 1 --2- 172.21.15.5:0/2612523411 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd3b806d300 0x7fd3b80b8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:45.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.248+0000 7fd3b67fc640 1 --2- 172.21.15.5:0/2612523411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3b80ad8e0 0x7fd3b81e39a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac00c4d0 tx=0x7fd3ac00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:45.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.248+0000 7fd39f7fe640 1 -- 172.21.15.5:0/2612523411 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac018070 con 0x7fd3b80ad8e0 2024-03-18T15:34:45.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.248+0000 7fd39f7fe640 1 -- 172.21.15.5:0/2612523411 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3ac0074e0 con 0x7fd3b80ad8e0 2024-03-18T15:34:45.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.248+0000 7fd3bd486640 1 -- 172.21.15.5:0/2612523411 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3b80b1560 con 0x7fd3b80ad8e0 2024-03-18T15:34:45.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.248+0000 7fd39f7fe640 1 -- 172.21.15.5:0/2612523411 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac004a30 con 0x7fd3b80ad8e0 2024-03-18T15:34:45.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.248+0000 7fd3bd486640 1 -- 172.21.15.5:0/2612523411 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3b80b1020 con 0x7fd3b80ad8e0 2024-03-18T15:34:45.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.248+0000 7fd3bd486640 1 -- 172.21.15.5:0/2612523411 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd3b804f5c0 con 0x7fd3b80ad8e0 2024-03-18T15:34:45.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.248+0000 7fd39f7fe640 1 -- 172.21.15.5:0/2612523411 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fd3ac01b070 con 0x7fd3b80ad8e0 2024-03-18T15:34:45.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.248+0000 7fd39f7fe640 1 --2- 172.21.15.5:0/2612523411 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd3a4067690 0x7fd3a4069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:45.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.248+0000 7fd39f7fe640 1 -- 172.21.15.5:0/2612523411 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7fd3ac084e00 con 0x7fd3b80ad8e0 2024-03-18T15:34:45.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.248+0000 7fd3b6ffd640 1 --2- 172.21.15.5:0/2612523411 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd3a4067690 0x7fd3a4069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:45.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.248+0000 7fd39f7fe640 1 --2- 172.21.15.5:0/2612523411 >> [v2:172.21.15.5:6816/502968448,v1:172.21.15.5:6817/502968448] conn(0x7fd3a406d550 0x7fd3a406f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:45.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.248+0000 7fd39f7fe640 1 -- 172.21.15.5:0/2612523411 --> [v2:172.21.15.5:6816/502968448,v1:172.21.15.5:6817/502968448] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd3a4070020 con 0x7fd3a406d550 2024-03-18T15:34:45.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.248+0000 7fd39f7fe640 1 -- 172.21.15.5:0/2612523411 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd3ac085290 con 0x7fd3b80ad8e0 2024-03-18T15:34:45.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.248+0000 7fd3b77fe640 1 --2- 172.21.15.5:0/2612523411 >> [v2:172.21.15.5:6816/502968448,v1:172.21.15.5:6817/502968448] conn(0x7fd3a406d550 0x7fd3a406f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:45.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.248+0000 7fd3b6ffd640 1 --2- 172.21.15.5:0/2612523411 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd3a4067690 0x7fd3a4069b50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd3a80ad000 tx=0x7fd3a80a4f90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:45.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.248+0000 7fd3b77fe640 1 --2- 172.21.15.5:0/2612523411 >> [v2:172.21.15.5:6816/502968448,v1:172.21.15.5:6817/502968448] conn(0x7fd3a406d550 0x7fd3a406f970 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:45.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.248+0000 7fd39f7fe640 1 -- 172.21.15.5:0/2612523411 <== osd.2 v2:172.21.15.5:6816/502968448 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd3a4070020 con 0x7fd3a406d550 2024-03-18T15:34:45.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.264+0000 7fd3bd486640 1 -- 172.21.15.5:0/2612523411 --> [v2:172.21.15.5:6816/502968448,v1:172.21.15.5:6817/502968448] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd3b81e3680 con 0x7fd3a406d550 2024-03-18T15:34:45.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.264+0000 7fd39f7fe640 1 -- 172.21.15.5:0/2612523411 <== osd.2 v2:172.21.15.5:6816/502968448 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd3b81e3680 con 0x7fd3a406d550 2024-03-18T15:34:45.267 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-03-18T15:34:45.267 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-18T15:34:45.267 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-03-18T15:34:45.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.264+0000 7fd3bd486640 1 -- 172.21.15.5:0/2612523411 >> [v2:172.21.15.5:6816/502968448,v1:172.21.15.5:6817/502968448] conn(0x7fd3a406d550 msgr2=0x7fd3a406f970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:45.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.264+0000 7fd3bd486640 1 --2- 172.21.15.5:0/2612523411 >> [v2:172.21.15.5:6816/502968448,v1:172.21.15.5:6817/502968448] conn(0x7fd3a406d550 0x7fd3a406f970 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:45.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.264+0000 7fd3bd486640 1 -- 172.21.15.5:0/2612523411 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd3a4067690 msgr2=0x7fd3a4069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:45.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.264+0000 7fd3bd486640 1 --2- 172.21.15.5:0/2612523411 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd3a4067690 0x7fd3a4069b50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd3a80ad000 tx=0x7fd3a80a4f90 comp rx=0 tx=0).stop 2024-03-18T15:34:45.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.264+0000 7fd3bd486640 1 -- 172.21.15.5:0/2612523411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3b80ad8e0 msgr2=0x7fd3b81e39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:45.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.264+0000 7fd3bd486640 1 --2- 172.21.15.5:0/2612523411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3b80ad8e0 0x7fd3b81e39a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac00c4d0 tx=0x7fd3ac00c990 comp rx=0 tx=0).stop 2024-03-18T15:34:45.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.264+0000 7fd3b77fe640 1 -- 172.21.15.5:0/2612523411 reap_dead start 2024-03-18T15:34:45.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.264+0000 7fd3bd486640 1 -- 172.21.15.5:0/2612523411 shutdown_connections 2024-03-18T15:34:45.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.264+0000 7fd3bd486640 1 -- 172.21.15.5:0/2612523411 >> 172.21.15.5:0/2612523411 conn(0x7fd3b80b5670 msgr2=0x7fd3b81473f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:45.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.264+0000 7fd3bd486640 1 -- 172.21.15.5:0/2612523411 shutdown_connections 2024-03-18T15:34:45.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.264+0000 7fd3bd486640 1 -- 172.21.15.5:0/2612523411 wait complete. 2024-03-18T15:34:45.282 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-03-18T15:34:45.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.400+0000 7f39ae1bf640 1 Processor -- start 2024-03-18T15:34:45.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.400+0000 7f39ae1bf640 1 -- start start 2024-03-18T15:34:45.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.400+0000 7f39ae1bf640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f39a813f260 0x7f39a813f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:45.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.400+0000 7f39ae1bf640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f39a814dc70 0x7f39a8150060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:45.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.400+0000 7f39ae1bf640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39a80b2ab0 con 0x7f39a806d2d0 2024-03-18T15:34:45.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.400+0000 7f39ae1bf640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f39a80728c0 con 0x7f39a814dc70 2024-03-18T15:34:45.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.400+0000 7f39ae1bf640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f39a80b0ea0 con 0x7f39a813f260 2024-03-18T15:34:45.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.400+0000 7f39a77fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f39a806d2d0 0x7f39a80b23a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:39876/0 (socket says 172.21.15.5:39876) 2024-03-18T15:34:45.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.400+0000 7f39a77fe640 1 -- 172.21.15.5:0/675041816 learned_addr learned my addr 172.21.15.5:0/675041816 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:45.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.400+0000 7f39a6ffd640 1 --2- 172.21.15.5:0/675041816 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f39a813f260 0x7f39a813f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:45.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.400+0000 7f39a7fff640 1 --2- 172.21.15.5:0/675041816 >> v2:172.21.15.80:3300/0 conn(0x7f39a814dc70 0x7f39a8150060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:45.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.400+0000 7f39a67fc640 1 -- 172.21.15.5:0/675041816 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 715691304 0 0) 0x7f39a80b2ab0 con 0x7f39a806d2d0 2024-03-18T15:34:45.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.400+0000 7f39a67fc640 1 -- 172.21.15.5:0/675041816 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3990003530 con 0x7f39a806d2d0 2024-03-18T15:34:45.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.400+0000 7f39a67fc640 1 -- 172.21.15.5:0/675041816 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 534032588 0 0) 0x7f3990003530 con 0x7f39a806d2d0 2024-03-18T15:34:45.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.400+0000 7f39a67fc640 1 -- 172.21.15.5:0/675041816 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f39a813f260 msgr2=0x7f39a813f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:45.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.400+0000 7f39a67fc640 1 --2- 172.21.15.5:0/675041816 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f39a813f260 0x7f39a813f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:45.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.400+0000 7f39a67fc640 1 -- 172.21.15.5:0/675041816 >> v2:172.21.15.80:3300/0 conn(0x7f39a814dc70 msgr2=0x7f39a8150060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:45.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.400+0000 7f39a67fc640 1 --2- 172.21.15.5:0/675041816 >> v2:172.21.15.80:3300/0 conn(0x7f39a814dc70 0x7f39a8150060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:45.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.400+0000 7f39a67fc640 1 -- 172.21.15.5:0/675041816 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a806d6b0 con 0x7f39a806d2d0 2024-03-18T15:34:45.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.400+0000 7f39a67fc640 1 -- 172.21.15.5:0/675041816 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3837549003 0 0) 0x7f3998001680 con 0x7f39a806d2d0 2024-03-18T15:34:45.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.400+0000 7f39a67fc640 1 -- 172.21.15.5:0/675041816 >> v1:172.21.15.5:6789/0 conn(0x7f39a806d2d0 legacy=0x7f39a80b23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:45.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.400+0000 7f39a67fc640 1 --2- 172.21.15.5:0/675041816 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3990003aa0 0x7f3990003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:45.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.400+0000 7f39a67fc640 1 --2- 172.21.15.5:0/675041816 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f39a813f260 0x7f3990004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:45.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.400+0000 7f39a67fc640 1 --2- 172.21.15.5:0/675041816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3990004d50 0x7f39900051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:45.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.400+0000 7f39a67fc640 1 -- 172.21.15.5:0/675041816 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f39a80728c0 con 0x7f3990004d50 2024-03-18T15:34:45.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.400+0000 7f39a67fc640 1 -- 172.21.15.5:0/675041816 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f39a80b0ea0 con 0x7f3990003aa0 2024-03-18T15:34:45.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.400+0000 7f39a67fc640 1 -- 172.21.15.5:0/675041816 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f399000baa0 con 0x7f39a813f260 2024-03-18T15:34:45.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.400+0000 7f39a7fff640 1 --2- 172.21.15.5:0/675041816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3990004d50 0x7f39900051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:45.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.400+0000 7f39a77fe640 1 --2- 172.21.15.5:0/675041816 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3990003aa0 0x7f3990003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:45.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.400+0000 7f39a6ffd640 1 --2- 172.21.15.5:0/675041816 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f39a813f260 0x7f3990004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:45.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.400+0000 7f39a7fff640 1 -- 172.21.15.5:0/675041816 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f39a813f260 msgr2=0x7f3990004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:45.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.400+0000 7f39a7fff640 1 --2- 172.21.15.5:0/675041816 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f39a813f260 0x7f3990004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:45.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.400+0000 7f39a7fff640 1 -- 172.21.15.5:0/675041816 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3990003aa0 msgr2=0x7f3990003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:45.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.400+0000 7f39a7fff640 1 --2- 172.21.15.5:0/675041816 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3990003aa0 0x7f3990003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:45.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.400+0000 7f39a7fff640 1 -- 172.21.15.5:0/675041816 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3990003530 con 0x7f3990004d50 2024-03-18T15:34:45.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.404+0000 7f39a7fff640 1 --2- 172.21.15.5:0/675041816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3990004d50 0x7f39900051d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f39a0009f90 tx=0x7f39a001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c98d9a3bb8f9d75 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:45.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.404+0000 7f39a67fc640 1 -- 172.21.15.5:0/675041816 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a00b2070 con 0x7f3990004d50 2024-03-18T15:34:45.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.404+0000 7f39a67fc640 1 -- 172.21.15.5:0/675041816 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39a00a5b80 con 0x7f3990004d50 2024-03-18T15:34:45.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.404+0000 7f39a67fc640 1 -- 172.21.15.5:0/675041816 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a0009c10 con 0x7f3990004d50 2024-03-18T15:34:45.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.404+0000 7f39ae1bf640 1 -- 172.21.15.5:0/675041816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3990004d50 msgr2=0x7f39900051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:45.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.404+0000 7f39ae1bf640 1 --2- 172.21.15.5:0/675041816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3990004d50 0x7f39900051d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f39a0009f90 tx=0x7f39a001cc80 comp rx=0 tx=0).stop 2024-03-18T15:34:45.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.404+0000 7f39a7fff640 1 -- 172.21.15.5:0/675041816 reap_dead start 2024-03-18T15:34:45.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.404+0000 7f39ae1bf640 1 -- 172.21.15.5:0/675041816 shutdown_connections 2024-03-18T15:34:45.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.404+0000 7f39ae1bf640 1 -- 172.21.15.5:0/675041816 >> 172.21.15.5:0/675041816 conn(0x7f39a80b5670 msgr2=0x7f39a80b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:45.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.404+0000 7f39ae1bf640 1 -- 172.21.15.5:0/675041816 shutdown_connections 2024-03-18T15:34:45.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.404+0000 7f39ae1bf640 1 -- 172.21.15.5:0/675041816 wait complete. 2024-03-18T15:34:45.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.404+0000 7f39ae1bf640 1 Processor -- start 2024-03-18T15:34:45.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.404+0000 7f39ae1bf640 1 -- start start 2024-03-18T15:34:45.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.408+0000 7f39ae1bf640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f39a8140490 0x7f39a8071b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:45.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.408+0000 7f39ae1bf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39a81ddbb0 0x7f39a81420b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:45.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.408+0000 7f39ae1bf640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39a81de280 0x7f39a81de700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:45.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.408+0000 7f39ae1bf640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f39a80b0070 con 0x7f39a81ddbb0 2024-03-18T15:34:45.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.408+0000 7f39ae1bf640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f39a80b1c30 con 0x7f39a81de280 2024-03-18T15:34:45.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.408+0000 7f39ae1bf640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f39a80b01f0 con 0x7f39a8140490 2024-03-18T15:34:45.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.408+0000 7f39a6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39a81ddbb0 0x7f39a81420b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:45.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.408+0000 7f39a6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39a81ddbb0 0x7f39a81420b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34126/0 (socket says 172.21.15.5:34126) 2024-03-18T15:34:45.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.408+0000 7f39a6ffd640 1 -- 172.21.15.5:0/2256857464 learned_addr learned my addr 172.21.15.5:0/2256857464 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:45.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.408+0000 7f39a77fe640 1 --2- 172.21.15.5:0/2256857464 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f39a8140490 0x7f39a8071b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:45.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.408+0000 7f39a7fff640 1 --2- 172.21.15.5:0/2256857464 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39a81de280 0x7f39a81de700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:45.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.408+0000 7f39a6ffd640 1 -- 172.21.15.5:0/2256857464 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f39a8140490 msgr2=0x7f39a8071b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:45.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.408+0000 7f39a6ffd640 1 --2- 172.21.15.5:0/2256857464 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f39a8140490 0x7f39a8071b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:45.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.408+0000 7f39a6ffd640 1 -- 172.21.15.5:0/2256857464 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39a81de280 msgr2=0x7f39a81de700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:45.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.408+0000 7f39a6ffd640 1 --2- 172.21.15.5:0/2256857464 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39a81de280 0x7f39a81de700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:45.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.408+0000 7f39a6ffd640 1 -- 172.21.15.5:0/2256857464 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a80720d0 con 0x7f39a81ddbb0 2024-03-18T15:34:45.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.408+0000 7f39a6ffd640 1 --2- 172.21.15.5:0/2256857464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39a81ddbb0 0x7f39a81420b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f399c011610 tx=0x7f399c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:45.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.408+0000 7f398ffff640 1 -- 172.21.15.5:0/2256857464 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399c00f8f0 con 0x7f39a81ddbb0 2024-03-18T15:34:45.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.408+0000 7f398ffff640 1 -- 172.21.15.5:0/2256857464 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f399c00fa90 con 0x7f39a81ddbb0 2024-03-18T15:34:45.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.408+0000 7f39ae1bf640 1 -- 172.21.15.5:0/2256857464 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39a8140870 con 0x7f39a81ddbb0 2024-03-18T15:34:45.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.408+0000 7f398ffff640 1 -- 172.21.15.5:0/2256857464 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399c00fe40 con 0x7f39a81ddbb0 2024-03-18T15:34:45.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.408+0000 7f39ae1bf640 1 -- 172.21.15.5:0/2256857464 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39a8071670 con 0x7f39a81ddbb0 2024-03-18T15:34:45.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.408+0000 7f39ae1bf640 1 -- 172.21.15.5:0/2256857464 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39a804f550 con 0x7f39a81ddbb0 2024-03-18T15:34:45.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.408+0000 7f398ffff640 1 -- 172.21.15.5:0/2256857464 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f399c015460 con 0x7f39a81ddbb0 2024-03-18T15:34:45.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.408+0000 7f398ffff640 1 --2- 172.21.15.5:0/2256857464 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3994067690 0x7f3994069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:45.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.408+0000 7f398ffff640 1 -- 172.21.15.5:0/2256857464 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7f399c085e40 con 0x7f39a81ddbb0 2024-03-18T15:34:45.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.408+0000 7f39a77fe640 1 --2- 172.21.15.5:0/2256857464 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3994067690 0x7f3994069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:45.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.408+0000 7f39a77fe640 1 --2- 172.21.15.5:0/2256857464 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3994067690 0x7f3994069b50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3998000c10 tx=0x7f3998009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:45.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.412+0000 7f398ffff640 1 -- 172.21.15.5:0/2256857464 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f399c019030 con 0x7f39a81ddbb0 2024-03-18T15:34:45.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.576+0000 7f39ae1bf640 1 -- 172.21.15.5:0/2256857464 --> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f39a813ec80 con 0x7f3994067690 2024-03-18T15:34:45.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.576+0000 7f398ffff640 1 -- 172.21.15.5:0/2256857464 <== mgr.4110 v2:172.21.15.5:6832/3362034860 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f39a813ec80 con 0x7f3994067690 2024-03-18T15:34:45.580 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 2 to deep-scrub 2024-03-18T15:34:45.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.580+0000 7f39ae1bf640 1 -- 172.21.15.5:0/2256857464 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3994067690 msgr2=0x7f3994069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:45.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.580+0000 7f39ae1bf640 1 --2- 172.21.15.5:0/2256857464 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3994067690 0x7f3994069b50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3998000c10 tx=0x7f3998009040 comp rx=0 tx=0).stop 2024-03-18T15:34:45.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.580+0000 7f39ae1bf640 1 -- 172.21.15.5:0/2256857464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39a81ddbb0 msgr2=0x7f39a81420b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:45.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.580+0000 7f39ae1bf640 1 --2- 172.21.15.5:0/2256857464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39a81ddbb0 0x7f39a81420b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f399c011610 tx=0x7f399c011ad0 comp rx=0 tx=0).stop 2024-03-18T15:34:45.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.580+0000 7f39ae1bf640 1 -- 172.21.15.5:0/2256857464 shutdown_connections 2024-03-18T15:34:45.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.580+0000 7f39ae1bf640 1 --2- 172.21.15.5:0/2256857464 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3994067690 0x7f3994069b50 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:45.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.580+0000 7f39ae1bf640 1 --2- 172.21.15.5:0/2256857464 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39a81de280 0x7f39a81de700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:45.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.580+0000 7f39ae1bf640 1 --2- 172.21.15.5:0/2256857464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39a81ddbb0 0x7f39a81420b0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:45.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.580+0000 7f39ae1bf640 1 --2- 172.21.15.5:0/2256857464 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f39a8140490 0x7f39a8071b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:45.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.580+0000 7f39ae1bf640 1 -- 172.21.15.5:0/2256857464 >> 172.21.15.5:0/2256857464 conn(0x7f39a80b5670 msgr2=0x7f39a806c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:45.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.580+0000 7f39ae1bf640 1 -- 172.21.15.5:0/2256857464 shutdown_connections 2024-03-18T15:34:45.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.580+0000 7f39ae1bf640 1 -- 172.21.15.5:0/2256857464 wait complete. 2024-03-18T15:34:45.598 INFO:tasks.ceph:Scrubbing osd.3 2024-03-18T15:34:45.598 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-03-18T15:34:45.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.724+0000 7f42c8779640 1 Processor -- start 2024-03-18T15:34:45.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.724+0000 7f42c8779640 1 -- start start 2024-03-18T15:34:45.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.724+0000 7f42c8779640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f42c00b56b0 0x7f42c014abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:45.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.724+0000 7f42c8779640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f42c014b130 0x7f42c0151f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:45.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.724+0000 7f42c8779640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42c014b560 con 0x7f42c00b7660 2024-03-18T15:34:45.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.724+0000 7f42c8779640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f42c006d800 con 0x7f42c00b56b0 2024-03-18T15:34:45.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.724+0000 7f42c8779640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f42c006dae0 con 0x7f42c014b130 2024-03-18T15:34:45.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.724+0000 7f42c6515640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f42c00b7660 0x7f42c0141510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:39878/0 (socket says 172.21.15.5:39878) 2024-03-18T15:34:45.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.724+0000 7f42c6515640 1 -- 172.21.15.5:0/1684755410 learned_addr learned my addr 172.21.15.5:0/1684755410 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:45.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.724+0000 7f42c5d14640 1 --2- 172.21.15.5:0/1684755410 >> v2:172.21.15.80:3300/0 conn(0x7f42c00b56b0 0x7f42c014abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:45.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.724+0000 7f42c6d16640 1 --2- 172.21.15.5:0/1684755410 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f42c014b130 0x7f42c0151f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:45.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.724+0000 7f42c5513640 1 -- 172.21.15.5:0/1684755410 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3702919925 0 0) 0x7f42c014b560 con 0x7f42c00b7660 2024-03-18T15:34:45.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.724+0000 7f42c5513640 1 -- 172.21.15.5:0/1684755410 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f42ac003530 con 0x7f42c00b7660 2024-03-18T15:34:45.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.724+0000 7f42c5513640 1 -- 172.21.15.5:0/1684755410 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3013718482 0 0) 0x7f42ac003530 con 0x7f42c00b7660 2024-03-18T15:34:45.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.724+0000 7f42c5513640 1 -- 172.21.15.5:0/1684755410 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f42c014b130 msgr2=0x7f42c0151f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:45.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.724+0000 7f42c5513640 1 --2- 172.21.15.5:0/1684755410 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f42c014b130 0x7f42c0151f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:45.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.724+0000 7f42c5513640 1 -- 172.21.15.5:0/1684755410 >> v2:172.21.15.80:3300/0 conn(0x7f42c00b56b0 msgr2=0x7f42c014abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:45.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.724+0000 7f42c5513640 1 --2- 172.21.15.5:0/1684755410 >> v2:172.21.15.80:3300/0 conn(0x7f42c00b56b0 0x7f42c014abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:45.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.724+0000 7f42c5513640 1 -- 172.21.15.5:0/1684755410 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42c00b7a40 con 0x7f42c00b7660 2024-03-18T15:34:45.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.724+0000 7f42c5513640 1 -- 172.21.15.5:0/1684755410 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3837549003 0 0) 0x7f42b4001680 con 0x7f42c00b7660 2024-03-18T15:34:45.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.724+0000 7f42c5513640 1 -- 172.21.15.5:0/1684755410 >> v1:172.21.15.5:6789/0 conn(0x7f42c00b7660 legacy=0x7f42c0141510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:45.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.724+0000 7f42c5513640 1 --2- 172.21.15.5:0/1684755410 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f42c014b130 0x7f42ac003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:45.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.724+0000 7f42c5513640 1 --2- 172.21.15.5:0/1684755410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42ac004020 0x7f42ac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:45.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.724+0000 7f42c5513640 1 --2- 172.21.15.5:0/1684755410 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42ac004d50 0x7f42ac0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:45.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.724+0000 7f42c5513640 1 -- 172.21.15.5:0/1684755410 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f42c006d800 con 0x7f42ac004020 2024-03-18T15:34:45.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.724+0000 7f42c5513640 1 -- 172.21.15.5:0/1684755410 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f42c006dae0 con 0x7f42ac004d50 2024-03-18T15:34:45.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.724+0000 7f42c5513640 1 -- 172.21.15.5:0/1684755410 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f42ac00bb00 con 0x7f42c014b130 2024-03-18T15:34:45.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.724+0000 7f42c5d14640 1 --2- 172.21.15.5:0/1684755410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42ac004020 0x7f42ac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:45.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.724+0000 7f42c6515640 1 --2- 172.21.15.5:0/1684755410 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f42c014b130 0x7f42ac003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:45.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.724+0000 7f42c6d16640 1 --2- 172.21.15.5:0/1684755410 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42ac004d50 0x7f42ac0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:45.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.724+0000 7f42c5d14640 1 -- 172.21.15.5:0/1684755410 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f42c014b130 msgr2=0x7f42ac003aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:45.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.724+0000 7f42c5d14640 1 --2- 172.21.15.5:0/1684755410 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f42c014b130 0x7f42ac003aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:45.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.724+0000 7f42c5d14640 1 -- 172.21.15.5:0/1684755410 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42ac004d50 msgr2=0x7f42ac0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:45.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.724+0000 7f42c5d14640 1 --2- 172.21.15.5:0/1684755410 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42ac004d50 0x7f42ac0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:45.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.724+0000 7f42c5d14640 1 -- 172.21.15.5:0/1684755410 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42ac003530 con 0x7f42ac004020 2024-03-18T15:34:45.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.728+0000 7f42c5d14640 1 --2- 172.21.15.5:0/1684755410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42ac004020 0x7f42ac004810 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f42bc009ae0 tx=0x7f42bc01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34db56fd04eb55a6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:45.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.728+0000 7f42c5513640 1 -- 172.21.15.5:0/1684755410 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42bc0096b0 con 0x7f42ac004020 2024-03-18T15:34:45.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.728+0000 7f42c5513640 1 -- 172.21.15.5:0/1684755410 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42bc0a6490 con 0x7f42ac004020 2024-03-18T15:34:45.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.728+0000 7f42c5513640 1 -- 172.21.15.5:0/1684755410 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42bc0a6880 con 0x7f42ac004020 2024-03-18T15:34:45.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.728+0000 7f42c8779640 1 -- 172.21.15.5:0/1684755410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42ac004020 msgr2=0x7f42ac004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:45.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.728+0000 7f42c8779640 1 --2- 172.21.15.5:0/1684755410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42ac004020 0x7f42ac004810 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f42bc009ae0 tx=0x7f42bc01cab0 comp rx=0 tx=0).stop 2024-03-18T15:34:45.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.728+0000 7f42c6d16640 1 -- 172.21.15.5:0/1684755410 reap_dead start 2024-03-18T15:34:45.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.728+0000 7f42c8779640 1 -- 172.21.15.5:0/1684755410 shutdown_connections 2024-03-18T15:34:45.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.728+0000 7f42c8779640 1 -- 172.21.15.5:0/1684755410 >> 172.21.15.5:0/1684755410 conn(0x7f42c0066fc0 msgr2=0x7f42c00673d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:45.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.728+0000 7f42c8779640 1 -- 172.21.15.5:0/1684755410 shutdown_connections 2024-03-18T15:34:45.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.728+0000 7f42c8779640 1 -- 172.21.15.5:0/1684755410 wait complete. 2024-03-18T15:34:45.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.728+0000 7f42c8779640 1 Processor -- start 2024-03-18T15:34:45.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.728+0000 7f42c8779640 1 -- start start 2024-03-18T15:34:45.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.732+0000 7f42c8779640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42ac004020 0x7f42c013df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:45.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.732+0000 7f42c8779640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42c00b34a0 0x7f42c00b49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:45.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.732+0000 7f42c8779640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f42c00b4ef0 0x7f42c0139ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:45.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.732+0000 7f42c8779640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f42c0152790 con 0x7f42ac004020 2024-03-18T15:34:45.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.732+0000 7f42c8779640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f42c0141e30 con 0x7f42c00b34a0 2024-03-18T15:34:45.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.732+0000 7f42c8779640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f42c0152910 con 0x7f42c00b4ef0 2024-03-18T15:34:45.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.732+0000 7f42c6515640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42ac004020 0x7f42c013df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:45.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.732+0000 7f42c6515640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42ac004020 0x7f42c013df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34154/0 (socket says 172.21.15.5:34154) 2024-03-18T15:34:45.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.732+0000 7f42c6515640 1 -- 172.21.15.5:0/2370044678 learned_addr learned my addr 172.21.15.5:0/2370044678 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:45.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.732+0000 7f42c5d14640 1 --2- 172.21.15.5:0/2370044678 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42c00b34a0 0x7f42c00b49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:45.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.732+0000 7f42c6515640 1 -- 172.21.15.5:0/2370044678 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f42c00b4ef0 msgr2=0x7f42c0139ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:45.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.732+0000 7f42c6d16640 1 --2- 172.21.15.5:0/2370044678 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f42c00b4ef0 0x7f42c0139ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:45.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.732+0000 7f42c6515640 1 --2- 172.21.15.5:0/2370044678 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f42c00b4ef0 0x7f42c0139ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:45.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.732+0000 7f42c6515640 1 -- 172.21.15.5:0/2370044678 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42c00b34a0 msgr2=0x7f42c00b49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:45.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.732+0000 7f42c6515640 1 --2- 172.21.15.5:0/2370044678 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42c00b34a0 0x7f42c00b49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:45.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.732+0000 7f42c6515640 1 -- 172.21.15.5:0/2370044678 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42c0148540 con 0x7f42ac004020 2024-03-18T15:34:45.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.732+0000 7f42c5d14640 1 --2- 172.21.15.5:0/2370044678 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42c00b34a0 0x7f42c00b49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:45.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.732+0000 7f42c6515640 1 --2- 172.21.15.5:0/2370044678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42ac004020 0x7f42c013df60 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f42b400e7b0 tx=0x7f42b400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:45.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.732+0000 7f42b2ffd640 1 -- 172.21.15.5:0/2370044678 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b400ee60 con 0x7f42ac004020 2024-03-18T15:34:45.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.732+0000 7f42b2ffd640 1 -- 172.21.15.5:0/2370044678 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42b4018070 con 0x7f42ac004020 2024-03-18T15:34:45.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.732+0000 7f42c8779640 1 -- 172.21.15.5:0/2370044678 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42c00b3c90 con 0x7f42ac004020 2024-03-18T15:34:45.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.732+0000 7f42b2ffd640 1 -- 172.21.15.5:0/2370044678 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b400cb10 con 0x7f42ac004020 2024-03-18T15:34:45.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.732+0000 7f42c8779640 1 -- 172.21.15.5:0/2370044678 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42c0145300 con 0x7f42ac004020 2024-03-18T15:34:45.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.732+0000 7f42c8779640 1 -- 172.21.15.5:0/2370044678 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f42c004f5c0 con 0x7f42ac004020 2024-03-18T15:34:45.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.732+0000 7f42b2ffd640 1 -- 172.21.15.5:0/2370044678 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f42b400ccb0 con 0x7f42ac004020 2024-03-18T15:34:45.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.732+0000 7f42b2ffd640 1 --2- 172.21.15.5:0/2370044678 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f42a4067660 0x7f42a4069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:45.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.732+0000 7f42b2ffd640 1 -- 172.21.15.5:0/2370044678 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7f42b4085e30 con 0x7f42ac004020 2024-03-18T15:34:45.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.732+0000 7f42c5d14640 1 --2- 172.21.15.5:0/2370044678 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f42a4067660 0x7f42a4069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:45.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.732+0000 7f42b2ffd640 1 --2- 172.21.15.5:0/2370044678 >> [v2:172.21.15.5:6824/2425932735,v1:172.21.15.5:6825/2425932735] conn(0x7f42a406d520 0x7f42a406f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:45.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.732+0000 7f42b2ffd640 1 -- 172.21.15.5:0/2370044678 --> [v2:172.21.15.5:6824/2425932735,v1:172.21.15.5:6825/2425932735] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f42a406fff0 con 0x7f42a406d520 2024-03-18T15:34:45.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.732+0000 7f42b2ffd640 1 -- 172.21.15.5:0/2370044678 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f42b40862c0 con 0x7f42ac004020 2024-03-18T15:34:45.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.732+0000 7f42c6d16640 1 --2- 172.21.15.5:0/2370044678 >> [v2:172.21.15.5:6824/2425932735,v1:172.21.15.5:6825/2425932735] conn(0x7f42a406d520 0x7f42a406f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:45.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.732+0000 7f42c5d14640 1 --2- 172.21.15.5:0/2370044678 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f42a4067660 0x7f42a4069b20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f42bc009ae0 tx=0x7f42bc0046d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:45.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.732+0000 7f42c6d16640 1 --2- 172.21.15.5:0/2370044678 >> [v2:172.21.15.5:6824/2425932735,v1:172.21.15.5:6825/2425932735] conn(0x7f42a406d520 0x7f42a406f940 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:45.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.736+0000 7f42b2ffd640 1 -- 172.21.15.5:0/2370044678 <== osd.3 v2:172.21.15.5:6824/2425932735 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f42a406fff0 con 0x7f42a406d520 2024-03-18T15:34:45.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.748+0000 7f42c8779640 1 -- 172.21.15.5:0/2370044678 --> [v2:172.21.15.5:6824/2425932735,v1:172.21.15.5:6825/2425932735] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f42c00b1920 con 0x7f42a406d520 2024-03-18T15:34:45.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.748+0000 7f42b2ffd640 1 -- 172.21.15.5:0/2370044678 <== osd.3 v2:172.21.15.5:6824/2425932735 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f42c00b1920 con 0x7f42a406d520 2024-03-18T15:34:45.751 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-03-18T15:34:45.751 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-18T15:34:45.751 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-03-18T15:34:45.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.748+0000 7f42c8779640 1 -- 172.21.15.5:0/2370044678 >> [v2:172.21.15.5:6824/2425932735,v1:172.21.15.5:6825/2425932735] conn(0x7f42a406d520 msgr2=0x7f42a406f940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:45.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.748+0000 7f42c8779640 1 --2- 172.21.15.5:0/2370044678 >> [v2:172.21.15.5:6824/2425932735,v1:172.21.15.5:6825/2425932735] conn(0x7f42a406d520 0x7f42a406f940 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:45.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.748+0000 7f42c8779640 1 -- 172.21.15.5:0/2370044678 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f42a4067660 msgr2=0x7f42a4069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:45.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.748+0000 7f42c8779640 1 --2- 172.21.15.5:0/2370044678 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f42a4067660 0x7f42a4069b20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f42bc009ae0 tx=0x7f42bc0046d0 comp rx=0 tx=0).stop 2024-03-18T15:34:45.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.748+0000 7f42c8779640 1 -- 172.21.15.5:0/2370044678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42ac004020 msgr2=0x7f42c013df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:45.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.748+0000 7f42c8779640 1 --2- 172.21.15.5:0/2370044678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42ac004020 0x7f42c013df60 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f42b400e7b0 tx=0x7f42b400ec70 comp rx=0 tx=0).stop 2024-03-18T15:34:45.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.748+0000 7f42c6d16640 1 -- 172.21.15.5:0/2370044678 reap_dead start 2024-03-18T15:34:45.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.748+0000 7f42c8779640 1 -- 172.21.15.5:0/2370044678 shutdown_connections 2024-03-18T15:34:45.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.748+0000 7f42c8779640 1 -- 172.21.15.5:0/2370044678 >> 172.21.15.5:0/2370044678 conn(0x7f42c0066fc0 msgr2=0x7f42c013ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:45.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.748+0000 7f42c8779640 1 -- 172.21.15.5:0/2370044678 shutdown_connections 2024-03-18T15:34:45.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.748+0000 7f42c8779640 1 -- 172.21.15.5:0/2370044678 wait complete. 2024-03-18T15:34:45.771 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-03-18T15:34:45.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.880+0000 7f8f0541f640 1 Processor -- start 2024-03-18T15:34:45.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.880+0000 7f8f0541f640 1 -- start start 2024-03-18T15:34:45.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.880+0000 7f8f0541f640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f8f0006b4e0 0x7f8f000702b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:45.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.880+0000 7f8f0541f640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f0006b8c0 con 0x7f8f00141360 2024-03-18T15:34:45.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.880+0000 7f8f0541f640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f0006c0e0 con 0x7f8f00147bd0 2024-03-18T15:34:45.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.880+0000 7f8f0541f640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f8f000719b0 con 0x7f8f0006b4e0 2024-03-18T15:34:45.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.880+0000 7f8efe7fc640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f8f00141360 0x7f8f001474c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:39894/0 (socket says 172.21.15.5:39894) 2024-03-18T15:34:45.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.880+0000 7f8efe7fc640 1 -- 172.21.15.5:0/302015325 learned_addr learned my addr 172.21.15.5:0/302015325 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:45.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.880+0000 7f8efeffd640 1 --2- 172.21.15.5:0/302015325 >> v2:172.21.15.80:3300/0 conn(0x7f8f0006b4e0 0x7f8f000702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:45.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.880+0000 7f8efdffb640 1 -- 172.21.15.5:0/302015325 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 190312506 0 0) 0x7f8f0006b8c0 con 0x7f8f00141360 2024-03-18T15:34:45.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.880+0000 7f8efdffb640 1 -- 172.21.15.5:0/302015325 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8eec003530 con 0x7f8f00141360 2024-03-18T15:34:45.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.880+0000 7f8efdffb640 1 -- 172.21.15.5:0/302015325 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4041523906 0 0) 0x7f8eec003530 con 0x7f8f00141360 2024-03-18T15:34:45.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.880+0000 7f8efdffb640 1 -- 172.21.15.5:0/302015325 >> v2:172.21.15.80:3300/0 conn(0x7f8f0006b4e0 msgr2=0x7f8f000702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:45.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.880+0000 7f8efdffb640 1 --2- 172.21.15.5:0/302015325 >> v2:172.21.15.80:3300/0 conn(0x7f8f0006b4e0 0x7f8f000702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:45.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.880+0000 7f8efdffb640 1 -- 172.21.15.5:0/302015325 >> v1:172.21.15.80:6789/0 conn(0x7f8f00147bd0 legacy=0x7f8f00151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:45.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.880+0000 7f8efdffb640 1 -- 172.21.15.5:0/302015325 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f0006e800 con 0x7f8f00141360 2024-03-18T15:34:45.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.880+0000 7f8efdffb640 1 -- 172.21.15.5:0/302015325 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3837549003 0 0) 0x7f8ee8001a00 con 0x7f8f00141360 2024-03-18T15:34:45.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.880+0000 7f8efdffb640 1 -- 172.21.15.5:0/302015325 >> v1:172.21.15.5:6789/0 conn(0x7f8f00141360 legacy=0x7f8f001474c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:45.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.880+0000 7f8efdffb640 1 --2- 172.21.15.5:0/302015325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8eec003aa0 0x7f8eec003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:45.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.880+0000 7f8efdffb640 1 --2- 172.21.15.5:0/302015325 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8eec004810 0x7f8eec004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:45.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.880+0000 7f8efdffb640 1 --2- 172.21.15.5:0/302015325 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8eec0052a0 0x7f8eec00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:45.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.880+0000 7f8efdffb640 1 -- 172.21.15.5:0/302015325 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8f000719b0 con 0x7f8eec003aa0 2024-03-18T15:34:45.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.880+0000 7f8efdffb640 1 -- 172.21.15.5:0/302015325 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8eec00c1f0 con 0x7f8eec0052a0 2024-03-18T15:34:45.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.880+0000 7f8efdffb640 1 -- 172.21.15.5:0/302015325 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f8eec00c3f0 con 0x7f8eec004810 2024-03-18T15:34:45.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.880+0000 7f8efeffd640 1 --2- 172.21.15.5:0/302015325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8eec003aa0 0x7f8eec003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:45.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.880+0000 7f8eff7fe640 1 --2- 172.21.15.5:0/302015325 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8eec0052a0 0x7f8eec00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:45.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.880+0000 7f8efe7fc640 1 --2- 172.21.15.5:0/302015325 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8eec004810 0x7f8eec004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:45.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.880+0000 7f8efeffd640 1 -- 172.21.15.5:0/302015325 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8eec004810 msgr2=0x7f8eec004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:45.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.880+0000 7f8efeffd640 1 --2- 172.21.15.5:0/302015325 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8eec004810 0x7f8eec004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:45.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.880+0000 7f8efeffd640 1 -- 172.21.15.5:0/302015325 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8eec0052a0 msgr2=0x7f8eec00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:45.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.880+0000 7f8efeffd640 1 --2- 172.21.15.5:0/302015325 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8eec0052a0 0x7f8eec00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:45.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.880+0000 7f8efeffd640 1 -- 172.21.15.5:0/302015325 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eec003530 con 0x7f8eec003aa0 2024-03-18T15:34:45.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.880+0000 7f8eff7fe640 1 --2- 172.21.15.5:0/302015325 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8eec0052a0 0x7f8eec00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:34:45.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.880+0000 7f8eff7fe640 1 -- 172.21.15.5:0/302015325 reap_dead start 2024-03-18T15:34:45.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.884+0000 7f8efeffd640 1 --2- 172.21.15.5:0/302015325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8eec003aa0 0x7f8eec003ec0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8009ae0 tx=0x7f8ef801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef42e51c649aab2 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:45.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.884+0000 7f8efdffb640 1 -- 172.21.15.5:0/302015325 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef80096b0 con 0x7f8eec003aa0 2024-03-18T15:34:45.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.884+0000 7f8efdffb640 1 -- 172.21.15.5:0/302015325 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ef80a6450 con 0x7f8eec003aa0 2024-03-18T15:34:45.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.884+0000 7f8efdffb640 1 -- 172.21.15.5:0/302015325 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef80a6840 con 0x7f8eec003aa0 2024-03-18T15:34:45.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.884+0000 7f8f0541f640 1 -- 172.21.15.5:0/302015325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8eec003aa0 msgr2=0x7f8eec003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:45.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.884+0000 7f8f0541f640 1 --2- 172.21.15.5:0/302015325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8eec003aa0 0x7f8eec003ec0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8009ae0 tx=0x7f8ef801ca70 comp rx=0 tx=0).stop 2024-03-18T15:34:45.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.884+0000 7f8f0541f640 1 -- 172.21.15.5:0/302015325 shutdown_connections 2024-03-18T15:34:45.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.884+0000 7f8f0541f640 1 --2- 172.21.15.5:0/302015325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8eec003aa0 0x7f8eec003ec0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:45.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.884+0000 7f8f0541f640 1 -- 172.21.15.5:0/302015325 >> 172.21.15.5:0/302015325 conn(0x7f8f000b7a80 msgr2=0x7f8f000b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:45.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.884+0000 7f8f0541f640 1 -- 172.21.15.5:0/302015325 shutdown_connections 2024-03-18T15:34:45.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.884+0000 7f8f0541f640 1 -- 172.21.15.5:0/302015325 wait complete. 2024-03-18T15:34:45.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.884+0000 7f8f0541f640 1 Processor -- start 2024-03-18T15:34:45.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.884+0000 7f8f0541f640 1 -- start start 2024-03-18T15:34:45.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.888+0000 7f8f0541f640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8eec003aa0 0x7f8f0013da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:45.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.888+0000 7f8f0541f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8f0013c910 0x7f8f0013cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:45.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.888+0000 7f8f0541f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f00148a40 0x7f8f00148ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:45.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.888+0000 7f8f0541f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8f00147640 con 0x7f8f00148a40 2024-03-18T15:34:45.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.888+0000 7f8f0541f640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8f001474c0 con 0x7f8f0013c910 2024-03-18T15:34:45.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.888+0000 7f8f0541f640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f8f001477c0 con 0x7f8eec003aa0 2024-03-18T15:34:45.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.888+0000 7f8eff7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f00148a40 0x7f8f00148ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:45.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.888+0000 7f8eff7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f00148a40 0x7f8f00148ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34174/0 (socket says 172.21.15.5:34174) 2024-03-18T15:34:45.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.888+0000 7f8eff7fe640 1 -- 172.21.15.5:0/282594866 learned_addr learned my addr 172.21.15.5:0/282594866 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:45.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.888+0000 7f8efeffd640 1 --2- 172.21.15.5:0/282594866 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8eec003aa0 0x7f8f0013da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:45.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.888+0000 7f8efe7fc640 1 --2- 172.21.15.5:0/282594866 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8f0013c910 0x7f8f0013cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:45.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.888+0000 7f8eff7fe640 1 -- 172.21.15.5:0/282594866 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8eec003aa0 msgr2=0x7f8f0013da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:45.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.888+0000 7f8eff7fe640 1 --2- 172.21.15.5:0/282594866 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8eec003aa0 0x7f8f0013da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:45.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.888+0000 7f8eff7fe640 1 -- 172.21.15.5:0/282594866 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8f0013c910 msgr2=0x7f8f0013cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:45.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.888+0000 7f8eff7fe640 1 --2- 172.21.15.5:0/282594866 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8f0013c910 0x7f8f0013cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:45.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.888+0000 7f8eff7fe640 1 -- 172.21.15.5:0/282594866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f0013d290 con 0x7f8f00148a40 2024-03-18T15:34:45.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.888+0000 7f8efe7fc640 1 --2- 172.21.15.5:0/282594866 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8f0013c910 0x7f8f0013cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:45.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.888+0000 7f8eff7fe640 1 --2- 172.21.15.5:0/282594866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f00148a40 0x7f8f00148ec0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8eec0057f0 tx=0x7f8ef000e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:45.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.888+0000 7f8ef77fe640 1 -- 172.21.15.5:0/282594866 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef0019070 con 0x7f8f00148a40 2024-03-18T15:34:45.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.888+0000 7f8ef77fe640 1 -- 172.21.15.5:0/282594866 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ef00052e0 con 0x7f8f00148a40 2024-03-18T15:34:45.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.888+0000 7f8f0541f640 1 -- 172.21.15.5:0/282594866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f001427e0 con 0x7f8f00148a40 2024-03-18T15:34:45.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.888+0000 7f8ef77fe640 1 -- 172.21.15.5:0/282594866 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef0003820 con 0x7f8f00148a40 2024-03-18T15:34:45.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.888+0000 7f8f0541f640 1 -- 172.21.15.5:0/282594866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f0014a100 con 0x7f8f00148a40 2024-03-18T15:34:45.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.888+0000 7f8f0541f640 1 -- 172.21.15.5:0/282594866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f0006c310 con 0x7f8f00148a40 2024-03-18T15:34:45.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.888+0000 7f8ef77fe640 1 -- 172.21.15.5:0/282594866 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f8ef001c070 con 0x7f8f00148a40 2024-03-18T15:34:45.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.888+0000 7f8ef77fe640 1 --2- 172.21.15.5:0/282594866 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f8ee0067690 0x7f8ee0069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:45.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.888+0000 7f8ef77fe640 1 -- 172.21.15.5:0/282594866 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7f8ef00849c0 con 0x7f8f00148a40 2024-03-18T15:34:45.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.888+0000 7f8efeffd640 1 --2- 172.21.15.5:0/282594866 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f8ee0067690 0x7f8ee0069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:45.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.888+0000 7f8efeffd640 1 --2- 172.21.15.5:0/282594866 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f8ee0067690 0x7f8ee0069b50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8ef80ad000 tx=0x7f8ef80046d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:45.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:45.892+0000 7f8ef77fe640 1 -- 172.21.15.5:0/282594866 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f8ef0086e20 con 0x7f8f00148a40 2024-03-18T15:34:46.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.048+0000 7f8f0541f640 1 -- 172.21.15.5:0/282594866 --> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f8f0004f550 con 0x7f8ee0067690 2024-03-18T15:34:46.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.052+0000 7f8ef77fe640 1 -- 172.21.15.5:0/282594866 <== mgr.4110 v2:172.21.15.5:6832/3362034860 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f8f0004f550 con 0x7f8ee0067690 2024-03-18T15:34:46.054 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 3 to deep-scrub 2024-03-18T15:34:46.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.052+0000 7f8f0541f640 1 -- 172.21.15.5:0/282594866 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f8ee0067690 msgr2=0x7f8ee0069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:46.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.052+0000 7f8f0541f640 1 --2- 172.21.15.5:0/282594866 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f8ee0067690 0x7f8ee0069b50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8ef80ad000 tx=0x7f8ef80046d0 comp rx=0 tx=0).stop 2024-03-18T15:34:46.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.052+0000 7f8f0541f640 1 -- 172.21.15.5:0/282594866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f00148a40 msgr2=0x7f8f00148ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:46.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.052+0000 7f8f0541f640 1 --2- 172.21.15.5:0/282594866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f00148a40 0x7f8f00148ec0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8eec0057f0 tx=0x7f8ef000e820 comp rx=0 tx=0).stop 2024-03-18T15:34:46.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.052+0000 7f8f0541f640 1 -- 172.21.15.5:0/282594866 shutdown_connections 2024-03-18T15:34:46.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.052+0000 7f8f0541f640 1 --2- 172.21.15.5:0/282594866 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f8ee0067690 0x7f8ee0069b50 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:46.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.052+0000 7f8f0541f640 1 --2- 172.21.15.5:0/282594866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f00148a40 0x7f8f00148ec0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:46.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.052+0000 7f8f0541f640 1 --2- 172.21.15.5:0/282594866 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8f0013c910 0x7f8f0013cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:46.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.052+0000 7f8f0541f640 1 --2- 172.21.15.5:0/282594866 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f8eec003aa0 0x7f8f0013da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:46.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.052+0000 7f8f0541f640 1 -- 172.21.15.5:0/282594866 >> 172.21.15.5:0/282594866 conn(0x7f8f000b7a80 msgr2=0x7f8f000b8780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:46.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.052+0000 7f8f0541f640 1 -- 172.21.15.5:0/282594866 shutdown_connections 2024-03-18T15:34:46.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.052+0000 7f8f0541f640 1 -- 172.21.15.5:0/282594866 wait complete. 2024-03-18T15:34:46.071 INFO:tasks.ceph:Scrubbing osd.4 2024-03-18T15:34:46.071 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-03-18T15:34:46.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.184+0000 7f5e53100640 1 Processor -- start 2024-03-18T15:34:46.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.184+0000 7f5e53100640 1 -- start start 2024-03-18T15:34:46.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.184+0000 7f5e53100640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5e4c141060 0x7f5e4c141510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:46.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.184+0000 7f5e53100640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5e4c14a9f0 0x7f5e4c14ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:46.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.184+0000 7f5e53100640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e4c0b52b0 con 0x7f5e4c0b9fb0 2024-03-18T15:34:46.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.184+0000 7f5e53100640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f5e4c0b5d50 con 0x7f5e4c14a9f0 2024-03-18T15:34:46.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.184+0000 7f5e53100640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f5e4c14b350 con 0x7f5e4c141060 2024-03-18T15:34:46.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.184+0000 7f5e5169d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5e4c14a9f0 0x7f5e4c14ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:46.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.184+0000 7f5e5169d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f5e4c14a9f0 0x7f5e4c14ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34188/0 (socket says 172.21.15.5:34188) 2024-03-18T15:34:46.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.184+0000 7f5e5169d640 1 -- 172.21.15.5:0/3253951803 learned_addr learned my addr 172.21.15.5:0/3253951803 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:46.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.184+0000 7f5e43fff640 1 --2- 172.21.15.5:0/3253951803 >> v2:172.21.15.80:3300/0 conn(0x7f5e4c141060 0x7f5e4c141510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:46.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.184+0000 7f5e5169d640 1 -- 172.21.15.5:0/3253951803 >> v2:172.21.15.80:3300/0 conn(0x7f5e4c141060 msgr2=0x7f5e4c141510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:46.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.184+0000 7f5e5169d640 1 --2- 172.21.15.5:0/3253951803 >> v2:172.21.15.80:3300/0 conn(0x7f5e4c141060 0x7f5e4c141510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:46.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.184+0000 7f5e5169d640 1 -- 172.21.15.5:0/3253951803 >> v1:172.21.15.80:6789/0 conn(0x7f5e4c0b9fb0 legacy=0x7f5e4c0b48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:46.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.184+0000 7f5e5169d640 1 -- 172.21.15.5:0/3253951803 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e4c0b5550 con 0x7f5e4c14a9f0 2024-03-18T15:34:46.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.188+0000 7f5e5169d640 1 --2- 172.21.15.5:0/3253951803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e4c14a9f0 0x7f5e4c14ae10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c0b7390 tx=0x7f5e4401c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5698b27e2b7bb37e server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:46.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.188+0000 7f5e437fe640 1 -- 172.21.15.5:0/3253951803 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e440a5790 con 0x7f5e4c14a9f0 2024-03-18T15:34:46.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.188+0000 7f5e437fe640 1 -- 172.21.15.5:0/3253951803 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e440a5930 con 0x7f5e4c14a9f0 2024-03-18T15:34:46.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.188+0000 7f5e437fe640 1 -- 172.21.15.5:0/3253951803 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e440a5d00 con 0x7f5e4c14a9f0 2024-03-18T15:34:46.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.188+0000 7f5e53100640 1 -- 172.21.15.5:0/3253951803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e4c14a9f0 msgr2=0x7f5e4c14ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:46.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.188+0000 7f5e53100640 1 --2- 172.21.15.5:0/3253951803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e4c14a9f0 0x7f5e4c14ae10 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c0b7390 tx=0x7f5e4401c8d0 comp rx=0 tx=0).stop 2024-03-18T15:34:46.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.188+0000 7f5e53100640 1 -- 172.21.15.5:0/3253951803 shutdown_connections 2024-03-18T15:34:46.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.188+0000 7f5e53100640 1 --2- 172.21.15.5:0/3253951803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e4c14a9f0 0x7f5e4c14ae10 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:46.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.188+0000 7f5e53100640 1 --2- 172.21.15.5:0/3253951803 >> v2:172.21.15.80:3300/0 conn(0x7f5e4c141060 0x7f5e4c141510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:46.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.188+0000 7f5e53100640 1 -- 172.21.15.5:0/3253951803 >> 172.21.15.5:0/3253951803 conn(0x7f5e4c0673f0 msgr2=0x7f5e4c067800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:46.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.188+0000 7f5e53100640 1 -- 172.21.15.5:0/3253951803 shutdown_connections 2024-03-18T15:34:46.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.188+0000 7f5e53100640 1 -- 172.21.15.5:0/3253951803 wait complete. 2024-03-18T15:34:46.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.188+0000 7f5e53100640 1 Processor -- start 2024-03-18T15:34:46.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.188+0000 7f5e53100640 1 -- start start 2024-03-18T15:34:46.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.192+0000 7f5e53100640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5e4c0b9fb0 0x7f5e4c138300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:46.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.192+0000 7f5e53100640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e4c141060 0x7f5e4c1dfb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:46.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.192+0000 7f5e53100640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5e4c14a9f0 0x7f5e4c133f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:46.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.192+0000 7f5e53100640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4c1545a0 con 0x7f5e4c141060 2024-03-18T15:34:46.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.192+0000 7f5e53100640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4c0b9330 con 0x7f5e4c14a9f0 2024-03-18T15:34:46.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.192+0000 7f5e53100640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f5e4c154720 con 0x7f5e4c0b9fb0 2024-03-18T15:34:46.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.192+0000 7f5e43fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e4c141060 0x7f5e4c1dfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:46.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.192+0000 7f5e43fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e4c141060 0x7f5e4c1dfb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34196/0 (socket says 172.21.15.5:34196) 2024-03-18T15:34:46.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.192+0000 7f5e43fff640 1 -- 172.21.15.5:0/480364249 learned_addr learned my addr 172.21.15.5:0/480364249 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:46.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.192+0000 7f5e50e9c640 1 --2- 172.21.15.5:0/480364249 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5e4c0b9fb0 0x7f5e4c138300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:46.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.192+0000 7f5e5169d640 1 --2- 172.21.15.5:0/480364249 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5e4c14a9f0 0x7f5e4c133f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:46.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.192+0000 7f5e43fff640 1 -- 172.21.15.5:0/480364249 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5e4c0b9fb0 msgr2=0x7f5e4c138300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:46.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.192+0000 7f5e43fff640 1 --2- 172.21.15.5:0/480364249 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5e4c0b9fb0 0x7f5e4c138300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:46.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.192+0000 7f5e43fff640 1 -- 172.21.15.5:0/480364249 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5e4c14a9f0 msgr2=0x7f5e4c133f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:46.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.192+0000 7f5e43fff640 1 --2- 172.21.15.5:0/480364249 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5e4c14a9f0 0x7f5e4c133f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:46.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.192+0000 7f5e43fff640 1 -- 172.21.15.5:0/480364249 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e4c1e0090 con 0x7f5e4c141060 2024-03-18T15:34:46.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.192+0000 7f5e5169d640 1 --2- 172.21.15.5:0/480364249 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5e4c14a9f0 0x7f5e4c133f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:46.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.192+0000 7f5e50e9c640 1 --2- 172.21.15.5:0/480364249 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f5e4c0b9fb0 0x7f5e4c138300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:46.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.192+0000 7f5e43fff640 1 --2- 172.21.15.5:0/480364249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e4c141060 0x7f5e4c1dfb50 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c00c7f0 tx=0x7f5e3c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:46.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.192+0000 7f5e417fa640 1 -- 172.21.15.5:0/480364249 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c015070 con 0x7f5e4c141060 2024-03-18T15:34:46.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.192+0000 7f5e417fa640 1 -- 172.21.15.5:0/480364249 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e3c004240 con 0x7f5e4c141060 2024-03-18T15:34:46.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.192+0000 7f5e53100640 1 -- 172.21.15.5:0/480364249 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e4c138cd0 con 0x7f5e4c141060 2024-03-18T15:34:46.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.192+0000 7f5e417fa640 1 -- 172.21.15.5:0/480364249 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c003620 con 0x7f5e4c141060 2024-03-18T15:34:46.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.192+0000 7f5e53100640 1 -- 172.21.15.5:0/480364249 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e4c139e20 con 0x7f5e4c141060 2024-03-18T15:34:46.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.192+0000 7f5e53100640 1 -- 172.21.15.5:0/480364249 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5e4c04f5c0 con 0x7f5e4c141060 2024-03-18T15:34:46.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.192+0000 7f5e417fa640 1 -- 172.21.15.5:0/480364249 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f5e3c0039d0 con 0x7f5e4c141060 2024-03-18T15:34:46.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.192+0000 7f5e417fa640 1 --2- 172.21.15.5:0/480364249 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5e38067690 0x7f5e38069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:46.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.192+0000 7f5e417fa640 1 -- 172.21.15.5:0/480364249 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7f5e3c085b60 con 0x7f5e4c141060 2024-03-18T15:34:46.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.192+0000 7f5e50e9c640 1 --2- 172.21.15.5:0/480364249 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5e38067690 0x7f5e38069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:46.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.192+0000 7f5e417fa640 1 --2- 172.21.15.5:0/480364249 >> [v2:172.21.15.80:6816/976514285,v1:172.21.15.80:6817/976514285] conn(0x7f5e3806d550 0x7f5e3806f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:46.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.192+0000 7f5e417fa640 1 -- 172.21.15.5:0/480364249 --> [v2:172.21.15.80:6816/976514285,v1:172.21.15.80:6817/976514285] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5e3c018ec0 con 0x7f5e3806d550 2024-03-18T15:34:46.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.192+0000 7f5e417fa640 1 -- 172.21.15.5:0/480364249 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5e3c085ff0 con 0x7f5e4c141060 2024-03-18T15:34:46.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.192+0000 7f5e50e9c640 1 --2- 172.21.15.5:0/480364249 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5e38067690 0x7f5e38069b50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5e48003e40 tx=0x7f5e48009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:46.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.192+0000 7f5e5169d640 1 --2- 172.21.15.5:0/480364249 >> [v2:172.21.15.80:6816/976514285,v1:172.21.15.80:6817/976514285] conn(0x7f5e3806d550 0x7f5e3806f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:46.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.192+0000 7f5e5169d640 1 --2- 172.21.15.5:0/480364249 >> [v2:172.21.15.80:6816/976514285,v1:172.21.15.80:6817/976514285] conn(0x7f5e3806d550 0x7f5e3806f970 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:46.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.196+0000 7f5e417fa640 1 -- 172.21.15.5:0/480364249 <== osd.4 v2:172.21.15.80:6816/976514285 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5e3c018ec0 con 0x7f5e3806d550 2024-03-18T15:34:46.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.208+0000 7f5e53100640 1 -- 172.21.15.5:0/480364249 --> [v2:172.21.15.80:6816/976514285,v1:172.21.15.80:6817/976514285] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f5e4c144fe0 con 0x7f5e3806d550 2024-03-18T15:34:46.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.208+0000 7f5e417fa640 1 -- 172.21.15.5:0/480364249 <== osd.4 v2:172.21.15.80:6816/976514285 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f5e4c144fe0 con 0x7f5e3806d550 2024-03-18T15:34:46.212 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-03-18T15:34:46.213 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-18T15:34:46.213 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-03-18T15:34:46.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.208+0000 7f5e53100640 1 -- 172.21.15.5:0/480364249 >> [v2:172.21.15.80:6816/976514285,v1:172.21.15.80:6817/976514285] conn(0x7f5e3806d550 msgr2=0x7f5e3806f970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:46.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.208+0000 7f5e53100640 1 --2- 172.21.15.5:0/480364249 >> [v2:172.21.15.80:6816/976514285,v1:172.21.15.80:6817/976514285] conn(0x7f5e3806d550 0x7f5e3806f970 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:46.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.208+0000 7f5e53100640 1 -- 172.21.15.5:0/480364249 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5e38067690 msgr2=0x7f5e38069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:46.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.208+0000 7f5e53100640 1 --2- 172.21.15.5:0/480364249 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f5e38067690 0x7f5e38069b50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5e48003e40 tx=0x7f5e48009040 comp rx=0 tx=0).stop 2024-03-18T15:34:46.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.208+0000 7f5e53100640 1 -- 172.21.15.5:0/480364249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e4c141060 msgr2=0x7f5e4c1dfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:46.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.208+0000 7f5e53100640 1 --2- 172.21.15.5:0/480364249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e4c141060 0x7f5e4c1dfb50 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c00c7f0 tx=0x7f5e3c00ccb0 comp rx=0 tx=0).stop 2024-03-18T15:34:46.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.208+0000 7f5e5169d640 1 -- 172.21.15.5:0/480364249 reap_dead start 2024-03-18T15:34:46.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.208+0000 7f5e53100640 1 -- 172.21.15.5:0/480364249 shutdown_connections 2024-03-18T15:34:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.208+0000 7f5e53100640 1 -- 172.21.15.5:0/480364249 >> 172.21.15.5:0/480364249 conn(0x7f5e4c0673f0 msgr2=0x7f5e4c0b8940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.208+0000 7f5e53100640 1 -- 172.21.15.5:0/480364249 shutdown_connections 2024-03-18T15:34:46.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.208+0000 7f5e53100640 1 -- 172.21.15.5:0/480364249 wait complete. 2024-03-18T15:34:46.227 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-03-18T15:34:46.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.336+0000 7f39a5524640 1 Processor -- start 2024-03-18T15:34:46.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.336+0000 7f39a5524640 1 -- start start 2024-03-18T15:34:46.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.336+0000 7f39a5524640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f39a00b7930 0x7f39a0141430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:46.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.336+0000 7f39a5524640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f39a0141970 0x7f39a00b68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:46.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.336+0000 7f39a5524640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39a006cc10 con 0x7f39a0072b20 2024-03-18T15:34:46.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.336+0000 7f39a5524640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f39a00b5e20 con 0x7f39a0141970 2024-03-18T15:34:46.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.336+0000 7f39a5524640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f39a00b7d10 con 0x7f39a00b7930 2024-03-18T15:34:46.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.336+0000 7f399effd640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f39a00b7930 0x7f39a0141430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:46.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.336+0000 7f399effd640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f39a00b7930 0x7f39a0141430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:55986/0 (socket says 172.21.15.5:55986) 2024-03-18T15:34:46.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.336+0000 7f399effd640 1 -- 172.21.15.5:0/2911289632 learned_addr learned my addr 172.21.15.5:0/2911289632 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:46.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.336+0000 7f399e7fc640 1 --2- 172.21.15.5:0/2911289632 >> v2:172.21.15.80:3300/0 conn(0x7f39a0141970 0x7f39a00b68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:46.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.336+0000 7f399effd640 1 -- 172.21.15.5:0/2911289632 >> v2:172.21.15.80:3300/0 conn(0x7f39a0141970 msgr2=0x7f39a00b68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:46.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.336+0000 7f399effd640 1 --2- 172.21.15.5:0/2911289632 >> v2:172.21.15.80:3300/0 conn(0x7f39a0141970 0x7f39a00b68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:46.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.336+0000 7f399effd640 1 -- 172.21.15.5:0/2911289632 >> v1:172.21.15.80:6789/0 conn(0x7f39a0072b20 legacy=0x7f39a0071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:46.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.336+0000 7f399effd640 1 -- 172.21.15.5:0/2911289632 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a00b6070 con 0x7f39a00b7930 2024-03-18T15:34:46.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.340+0000 7f399effd640 1 --2- 172.21.15.5:0/2911289632 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f39a00b7930 0x7f39a0141430 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3998009e50 tx=0x7f399801c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=809d48d5a9be9daf server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:46.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.340+0000 7f399dffb640 1 -- 172.21.15.5:0/2911289632 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39980a5760 con 0x7f39a00b7930 2024-03-18T15:34:46.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.340+0000 7f399dffb640 1 -- 172.21.15.5:0/2911289632 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39980a5900 con 0x7f39a00b7930 2024-03-18T15:34:46.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.340+0000 7f399dffb640 1 -- 172.21.15.5:0/2911289632 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39980a5cd0 con 0x7f39a00b7930 2024-03-18T15:34:46.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.340+0000 7f39a5524640 1 -- 172.21.15.5:0/2911289632 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f39a00b7930 msgr2=0x7f39a0141430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:46.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.340+0000 7f39a5524640 1 --2- 172.21.15.5:0/2911289632 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f39a00b7930 0x7f39a0141430 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3998009e50 tx=0x7f399801c8e0 comp rx=0 tx=0).stop 2024-03-18T15:34:46.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.340+0000 7f39a5524640 1 -- 172.21.15.5:0/2911289632 shutdown_connections 2024-03-18T15:34:46.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.340+0000 7f39a5524640 1 --2- 172.21.15.5:0/2911289632 >> v2:172.21.15.80:3300/0 conn(0x7f39a0141970 0x7f39a00b68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:46.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.340+0000 7f39a5524640 1 --2- 172.21.15.5:0/2911289632 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f39a00b7930 0x7f39a0141430 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:46.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.340+0000 7f39a5524640 1 -- 172.21.15.5:0/2911289632 >> 172.21.15.5:0/2911289632 conn(0x7f39a0066fc0 msgr2=0x7f39a00673d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:46.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.340+0000 7f39a5524640 1 -- 172.21.15.5:0/2911289632 shutdown_connections 2024-03-18T15:34:46.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.340+0000 7f39a5524640 1 -- 172.21.15.5:0/2911289632 wait complete. 2024-03-18T15:34:46.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.344+0000 7f39a5524640 1 Processor -- start 2024-03-18T15:34:46.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.344+0000 7f39a5524640 1 -- start start 2024-03-18T15:34:46.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.344+0000 7f39a5524640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39a0072b20 0x7f39a01463f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:46.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.344+0000 7f39a5524640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f39a00b7930 0x7f39a0139bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:46.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.344+0000 7f39a5524640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39a0141970 0x7f39a013a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:46.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.344+0000 7f39a5524640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f39a00b5a50 con 0x7f39a0072b20 2024-03-18T15:34:46.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.344+0000 7f39a5524640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f39a00b58d0 con 0x7f39a0141970 2024-03-18T15:34:46.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.344+0000 7f39a5524640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f39a0154d90 con 0x7f39a00b7930 2024-03-18T15:34:46.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.344+0000 7f399effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39a0072b20 0x7f39a01463f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:46.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.344+0000 7f399effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39a0072b20 0x7f39a01463f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34200/0 (socket says 172.21.15.5:34200) 2024-03-18T15:34:46.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.344+0000 7f399effd640 1 -- 172.21.15.5:0/2841455557 learned_addr learned my addr 172.21.15.5:0/2841455557 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:46.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.344+0000 7f399f7fe640 1 --2- 172.21.15.5:0/2841455557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39a0141970 0x7f39a013a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:46.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.344+0000 7f399e7fc640 1 --2- 172.21.15.5:0/2841455557 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f39a00b7930 0x7f39a0139bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:46.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.344+0000 7f399effd640 1 -- 172.21.15.5:0/2841455557 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f39a00b7930 msgr2=0x7f39a0139bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:46.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.344+0000 7f399effd640 1 --2- 172.21.15.5:0/2841455557 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f39a00b7930 0x7f39a0139bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:46.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.344+0000 7f399effd640 1 -- 172.21.15.5:0/2841455557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39a0141970 msgr2=0x7f39a013a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:46.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.344+0000 7f399effd640 1 --2- 172.21.15.5:0/2841455557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39a0141970 0x7f39a013a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:46.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.344+0000 7f399effd640 1 -- 172.21.15.5:0/2841455557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a0134ce0 con 0x7f39a0072b20 2024-03-18T15:34:46.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.344+0000 7f399e7fc640 1 --2- 172.21.15.5:0/2841455557 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f39a00b7930 0x7f39a0139bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:34:46.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.344+0000 7f399effd640 1 --2- 172.21.15.5:0/2841455557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39a0072b20 0x7f39a01463f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f399801a510 tx=0x7f3998019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:46.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.344+0000 7f39877fe640 1 -- 172.21.15.5:0/2841455557 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39980bc020 con 0x7f39a0072b20 2024-03-18T15:34:46.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.344+0000 7f39877fe640 1 -- 172.21.15.5:0/2841455557 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39980b8690 con 0x7f39a0072b20 2024-03-18T15:34:46.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.344+0000 7f39a5524640 1 -- 172.21.15.5:0/2841455557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39a0146d80 con 0x7f39a0072b20 2024-03-18T15:34:46.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.344+0000 7f39877fe640 1 -- 172.21.15.5:0/2841455557 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39980b8a80 con 0x7f39a0072b20 2024-03-18T15:34:46.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.344+0000 7f39a5524640 1 -- 172.21.15.5:0/2841455557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39a00b9480 con 0x7f39a0072b20 2024-03-18T15:34:46.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.344+0000 7f39a5524640 1 -- 172.21.15.5:0/2841455557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39a004f550 con 0x7f39a0072b20 2024-03-18T15:34:46.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.348+0000 7f39877fe640 1 -- 172.21.15.5:0/2841455557 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f39980bf020 con 0x7f39a0072b20 2024-03-18T15:34:46.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.348+0000 7f39877fe640 1 --2- 172.21.15.5:0/2841455557 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f398c067660 0x7f398c069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:46.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.348+0000 7f39877fe640 1 -- 172.21.15.5:0/2841455557 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7f39981201f0 con 0x7f39a0072b20 2024-03-18T15:34:46.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.348+0000 7f399e7fc640 1 --2- 172.21.15.5:0/2841455557 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f398c067660 0x7f398c069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:46.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.348+0000 7f399e7fc640 1 --2- 172.21.15.5:0/2841455557 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f398c067660 0x7f398c069b20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f39a0145070 tx=0x7f3994008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:46.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.352+0000 7f39877fe640 1 -- 172.21.15.5:0/2841455557 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f39980c6020 con 0x7f39a0072b20 2024-03-18T15:34:46.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.504+0000 7f39a5524640 1 -- 172.21.15.5:0/2841455557 --> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f39a0147f50 con 0x7f398c067660 2024-03-18T15:34:46.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.508+0000 7f39877fe640 1 -- 172.21.15.5:0/2841455557 <== mgr.4110 v2:172.21.15.5:6832/3362034860 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f39a0147f50 con 0x7f398c067660 2024-03-18T15:34:46.509 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 4 to deep-scrub 2024-03-18T15:34:46.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.508+0000 7f39a5524640 1 -- 172.21.15.5:0/2841455557 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f398c067660 msgr2=0x7f398c069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:46.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.508+0000 7f39a5524640 1 --2- 172.21.15.5:0/2841455557 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f398c067660 0x7f398c069b20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f39a0145070 tx=0x7f3994008040 comp rx=0 tx=0).stop 2024-03-18T15:34:46.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.508+0000 7f39a5524640 1 -- 172.21.15.5:0/2841455557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39a0072b20 msgr2=0x7f39a01463f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:46.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.508+0000 7f39a5524640 1 --2- 172.21.15.5:0/2841455557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39a0072b20 0x7f39a01463f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f399801a510 tx=0x7f3998019680 comp rx=0 tx=0).stop 2024-03-18T15:34:46.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.508+0000 7f39a5524640 1 -- 172.21.15.5:0/2841455557 shutdown_connections 2024-03-18T15:34:46.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.508+0000 7f39a5524640 1 --2- 172.21.15.5:0/2841455557 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f398c067660 0x7f398c069b20 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:46.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.508+0000 7f39a5524640 1 --2- 172.21.15.5:0/2841455557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39a0141970 0x7f39a013a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:46.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.508+0000 7f39a5524640 1 --2- 172.21.15.5:0/2841455557 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f39a00b7930 0x7f39a0139bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:46.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.508+0000 7f39a5524640 1 --2- 172.21.15.5:0/2841455557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39a0072b20 0x7f39a01463f0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:46.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.508+0000 7f39a5524640 1 -- 172.21.15.5:0/2841455557 >> 172.21.15.5:0/2841455557 conn(0x7f39a0066fc0 msgr2=0x7f39a013e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:46.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.508+0000 7f39a5524640 1 -- 172.21.15.5:0/2841455557 shutdown_connections 2024-03-18T15:34:46.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.508+0000 7f39a5524640 1 -- 172.21.15.5:0/2841455557 wait complete. 2024-03-18T15:34:46.527 INFO:tasks.ceph:Scrubbing osd.5 2024-03-18T15:34:46.527 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-03-18T15:34:46.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.652+0000 7f2962285640 1 Processor -- start 2024-03-18T15:34:46.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.652+0000 7f2962285640 1 -- start start 2024-03-18T15:34:46.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.652+0000 7f2962285640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f295c072170 0x7f295c072550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:46.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.652+0000 7f2962285640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f295c1483f0 con 0x7f295c06b280 2024-03-18T15:34:46.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.652+0000 7f2962285640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f295c071710 con 0x7f295c141970 2024-03-18T15:34:46.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.652+0000 7f2962285640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f295c147910 con 0x7f295c072170 2024-03-18T15:34:46.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.656+0000 7f295b7fe640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f295c06b280 0x7f295c141260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:39904/0 (socket says 172.21.15.5:39904) 2024-03-18T15:34:46.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.656+0000 7f295b7fe640 1 -- 172.21.15.5:0/840811495 learned_addr learned my addr 172.21.15.5:0/840811495 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:46.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.656+0000 7f295bfff640 1 --2- 172.21.15.5:0/840811495 >> v2:172.21.15.80:3300/0 conn(0x7f295c072170 0x7f295c072550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:46.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.656+0000 7f295affd640 1 -- 172.21.15.5:0/840811495 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1053385262 0 0) 0x7f295c1483f0 con 0x7f295c06b280 2024-03-18T15:34:46.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.656+0000 7f295affd640 1 -- 172.21.15.5:0/840811495 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2944003530 con 0x7f295c06b280 2024-03-18T15:34:46.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.656+0000 7f295affd640 1 -- 172.21.15.5:0/840811495 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2798320802 0 0) 0x7f2944003530 con 0x7f295c06b280 2024-03-18T15:34:46.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.656+0000 7f295affd640 1 -- 172.21.15.5:0/840811495 >> v2:172.21.15.80:3300/0 conn(0x7f295c072170 msgr2=0x7f295c072550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:46.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.656+0000 7f295affd640 1 --2- 172.21.15.5:0/840811495 >> v2:172.21.15.80:3300/0 conn(0x7f295c072170 0x7f295c072550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:46.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.656+0000 7f295affd640 1 -- 172.21.15.5:0/840811495 >> v1:172.21.15.80:6789/0 conn(0x7f295c141970 legacy=0x7f295c151d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:46.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.656+0000 7f295affd640 1 -- 172.21.15.5:0/840811495 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f295c147bf0 con 0x7f295c06b280 2024-03-18T15:34:46.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.656+0000 7f295affd640 1 -- 172.21.15.5:0/840811495 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3837549003 0 0) 0x7f2950001680 con 0x7f295c06b280 2024-03-18T15:34:46.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.656+0000 7f295affd640 1 -- 172.21.15.5:0/840811495 >> v1:172.21.15.5:6789/0 conn(0x7f295c06b280 legacy=0x7f295c141260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:46.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.656+0000 7f295affd640 1 --2- 172.21.15.5:0/840811495 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2944003aa0 0x7f2944003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:46.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.656+0000 7f295affd640 1 --2- 172.21.15.5:0/840811495 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2944004810 0x7f2944004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:46.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.656+0000 7f295affd640 1 --2- 172.21.15.5:0/840811495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2944005220 0x7f294400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:46.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.656+0000 7f295affd640 1 -- 172.21.15.5:0/840811495 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f295c147910 con 0x7f2944005220 2024-03-18T15:34:46.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.656+0000 7f295affd640 1 -- 172.21.15.5:0/840811495 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f294400c150 con 0x7f2944003aa0 2024-03-18T15:34:46.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.656+0000 7f295affd640 1 -- 172.21.15.5:0/840811495 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f294400c350 con 0x7f2944004810 2024-03-18T15:34:46.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.656+0000 7f2960822640 1 --2- 172.21.15.5:0/840811495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2944005220 0x7f294400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:46.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.656+0000 7f295bfff640 1 --2- 172.21.15.5:0/840811495 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2944003aa0 0x7f2944003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:46.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.656+0000 7f295b7fe640 1 --2- 172.21.15.5:0/840811495 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2944004810 0x7f2944004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:46.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.656+0000 7f2960822640 1 -- 172.21.15.5:0/840811495 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2944004810 msgr2=0x7f2944004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:46.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.656+0000 7f2960822640 1 --2- 172.21.15.5:0/840811495 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2944004810 0x7f2944004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:46.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.656+0000 7f2960822640 1 -- 172.21.15.5:0/840811495 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2944003aa0 msgr2=0x7f2944003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:46.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.656+0000 7f2960822640 1 --2- 172.21.15.5:0/840811495 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2944003aa0 0x7f2944003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:46.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.656+0000 7f2960822640 1 -- 172.21.15.5:0/840811495 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2944003530 con 0x7f2944005220 2024-03-18T15:34:46.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.656+0000 7f295bfff640 1 --2- 172.21.15.5:0/840811495 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2944003aa0 0x7f2944003ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:34:46.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.656+0000 7f2960822640 1 -- 172.21.15.5:0/840811495 reap_dead start 2024-03-18T15:34:46.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.656+0000 7f2960822640 1 --2- 172.21.15.5:0/840811495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2944005220 0x7f294400b9d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f295c141e00 tx=0x7f295401c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c300bfb3c068e5c server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:46.665 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.656+0000 7f295affd640 1 -- 172.21.15.5:0/840811495 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2954001a90 con 0x7f2944005220 2024-03-18T15:34:46.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.656+0000 7f295affd640 1 -- 172.21.15.5:0/840811495 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2954003b80 con 0x7f2944005220 2024-03-18T15:34:46.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.656+0000 7f295affd640 1 -- 172.21.15.5:0/840811495 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2954005020 con 0x7f2944005220 2024-03-18T15:34:46.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.660+0000 7f2962285640 1 -- 172.21.15.5:0/840811495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2944005220 msgr2=0x7f294400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:46.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.660+0000 7f2962285640 1 --2- 172.21.15.5:0/840811495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2944005220 0x7f294400b9d0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f295c141e00 tx=0x7f295401c720 comp rx=0 tx=0).stop 2024-03-18T15:34:46.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.660+0000 7f2962285640 1 -- 172.21.15.5:0/840811495 shutdown_connections 2024-03-18T15:34:46.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.660+0000 7f2962285640 1 --2- 172.21.15.5:0/840811495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2944005220 0x7f294400b9d0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:46.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.660+0000 7f2962285640 1 -- 172.21.15.5:0/840811495 >> 172.21.15.5:0/840811495 conn(0x7f295c0b6680 msgr2=0x7f295c0b6a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:46.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.660+0000 7f2962285640 1 -- 172.21.15.5:0/840811495 shutdown_connections 2024-03-18T15:34:46.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.660+0000 7f2962285640 1 -- 172.21.15.5:0/840811495 wait complete. 2024-03-18T15:34:46.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.660+0000 7f2962285640 1 Processor -- start 2024-03-18T15:34:46.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.660+0000 7f2962285640 1 -- start start 2024-03-18T15:34:46.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.660+0000 7f2962285640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2944005220 0x7f295c149ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:46.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.660+0000 7f2962285640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f295c14b3c0 0x7f295c1df8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:46.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.660+0000 7f2962285640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f295c148cd0 0x7f295c1dfe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:46.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.660+0000 7f2962285640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f295c0709c0 con 0x7f295c14b3c0 2024-03-18T15:34:46.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.660+0000 7f2962285640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f295c070840 con 0x7f295c148cd0 2024-03-18T15:34:46.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.660+0000 7f2962285640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f295c1474c0 con 0x7f2944005220 2024-03-18T15:34:46.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.660+0000 7f295b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f295c14b3c0 0x7f295c1df8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:46.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.660+0000 7f295b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f295c14b3c0 0x7f295c1df8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34222/0 (socket says 172.21.15.5:34222) 2024-03-18T15:34:46.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.660+0000 7f295b7fe640 1 -- 172.21.15.5:0/3783637227 learned_addr learned my addr 172.21.15.5:0/3783637227 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:46.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.660+0000 7f295bfff640 1 --2- 172.21.15.5:0/3783637227 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2944005220 0x7f295c149ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:46.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.660+0000 7f2960822640 1 --2- 172.21.15.5:0/3783637227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f295c148cd0 0x7f295c1dfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:46.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.660+0000 7f295b7fe640 1 -- 172.21.15.5:0/3783637227 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2944005220 msgr2=0x7f295c149ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:46.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.660+0000 7f295b7fe640 1 --2- 172.21.15.5:0/3783637227 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f2944005220 0x7f295c149ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:46.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.660+0000 7f295b7fe640 1 -- 172.21.15.5:0/3783637227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f295c148cd0 msgr2=0x7f295c1dfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:46.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.660+0000 7f295b7fe640 1 --2- 172.21.15.5:0/3783637227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f295c148cd0 0x7f295c1dfe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:46.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.660+0000 7f295b7fe640 1 -- 172.21.15.5:0/3783637227 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f295c0b3ac0 con 0x7f295c14b3c0 2024-03-18T15:34:46.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.660+0000 7f2960822640 1 --2- 172.21.15.5:0/3783637227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f295c148cd0 0x7f295c1dfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:46.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.660+0000 7f295b7fe640 1 --2- 172.21.15.5:0/3783637227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f295c14b3c0 0x7f295c1df8f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f295000e780 tx=0x7f295000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:46.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.660+0000 7f2958ff9640 1 -- 172.21.15.5:0/3783637227 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f295000c6c0 con 0x7f295c14b3c0 2024-03-18T15:34:46.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.660+0000 7f2958ff9640 1 -- 172.21.15.5:0/3783637227 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2950018070 con 0x7f295c14b3c0 2024-03-18T15:34:46.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.660+0000 7f2958ff9640 1 -- 172.21.15.5:0/3783637227 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f295000caa0 con 0x7f295c14b3c0 2024-03-18T15:34:46.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.660+0000 7f2962285640 1 -- 172.21.15.5:0/3783637227 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f295c1491f0 con 0x7f295c14b3c0 2024-03-18T15:34:46.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.660+0000 7f2962285640 1 -- 172.21.15.5:0/3783637227 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f295c1497b0 con 0x7f295c14b3c0 2024-03-18T15:34:46.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.664+0000 7f2962285640 1 -- 172.21.15.5:0/3783637227 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f295c04f5c0 con 0x7f295c14b3c0 2024-03-18T15:34:46.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.664+0000 7f2958ff9640 1 -- 172.21.15.5:0/3783637227 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f295000cc40 con 0x7f295c14b3c0 2024-03-18T15:34:46.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.664+0000 7f2958ff9640 1 --2- 172.21.15.5:0/3783637227 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f2948067690 0x7f2948069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:46.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.664+0000 7f2958ff9640 1 -- 172.21.15.5:0/3783637227 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7f2950085980 con 0x7f295c14b3c0 2024-03-18T15:34:46.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.664+0000 7f295bfff640 1 --2- 172.21.15.5:0/3783637227 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f2948067690 0x7f2948069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:46.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.664+0000 7f2958ff9640 1 --2- 172.21.15.5:0/3783637227 >> [v2:172.21.15.80:6824/627153447,v1:172.21.15.80:6825/627153447] conn(0x7f294806d550 0x7f294806f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:46.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.664+0000 7f2958ff9640 1 -- 172.21.15.5:0/3783637227 --> [v2:172.21.15.80:6824/627153447,v1:172.21.15.80:6825/627153447] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2950018e50 con 0x7f294806d550 2024-03-18T15:34:46.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.664+0000 7f2958ff9640 1 -- 172.21.15.5:0/3783637227 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2950085d30 con 0x7f295c14b3c0 2024-03-18T15:34:46.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.664+0000 7f2960822640 1 --2- 172.21.15.5:0/3783637227 >> [v2:172.21.15.80:6824/627153447,v1:172.21.15.80:6825/627153447] conn(0x7f294806d550 0x7f294806f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:46.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.664+0000 7f295bfff640 1 --2- 172.21.15.5:0/3783637227 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f2948067690 0x7f2948069b50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f294c0099b0 tx=0x7f294c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:46.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.664+0000 7f2960822640 1 --2- 172.21.15.5:0/3783637227 >> [v2:172.21.15.80:6824/627153447,v1:172.21.15.80:6825/627153447] conn(0x7f294806d550 0x7f294806f970 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:46.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.668+0000 7f2958ff9640 1 -- 172.21.15.5:0/3783637227 <== osd.5 v2:172.21.15.80:6824/627153447 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2950018e50 con 0x7f294806d550 2024-03-18T15:34:46.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.680+0000 7f2962285640 1 -- 172.21.15.5:0/3783637227 --> [v2:172.21.15.80:6824/627153447,v1:172.21.15.80:6825/627153447] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f295c06b660 con 0x7f294806d550 2024-03-18T15:34:46.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.680+0000 7f2958ff9640 1 -- 172.21.15.5:0/3783637227 <== osd.5 v2:172.21.15.80:6824/627153447 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f295c06b660 con 0x7f294806d550 2024-03-18T15:34:46.683 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-03-18T15:34:46.683 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-18T15:34:46.684 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-03-18T15:34:46.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.680+0000 7f2962285640 1 -- 172.21.15.5:0/3783637227 >> [v2:172.21.15.80:6824/627153447,v1:172.21.15.80:6825/627153447] conn(0x7f294806d550 msgr2=0x7f294806f970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:46.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.680+0000 7f2962285640 1 --2- 172.21.15.5:0/3783637227 >> [v2:172.21.15.80:6824/627153447,v1:172.21.15.80:6825/627153447] conn(0x7f294806d550 0x7f294806f970 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:46.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.680+0000 7f2962285640 1 -- 172.21.15.5:0/3783637227 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f2948067690 msgr2=0x7f2948069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:46.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.680+0000 7f2962285640 1 --2- 172.21.15.5:0/3783637227 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f2948067690 0x7f2948069b50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f294c0099b0 tx=0x7f294c008040 comp rx=0 tx=0).stop 2024-03-18T15:34:46.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.680+0000 7f2962285640 1 -- 172.21.15.5:0/3783637227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f295c14b3c0 msgr2=0x7f295c1df8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:46.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.680+0000 7f2962285640 1 --2- 172.21.15.5:0/3783637227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f295c14b3c0 0x7f295c1df8f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f295000e780 tx=0x7f295000ec40 comp rx=0 tx=0).stop 2024-03-18T15:34:46.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.680+0000 7f2960822640 1 -- 172.21.15.5:0/3783637227 reap_dead start 2024-03-18T15:34:46.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.680+0000 7f2962285640 1 -- 172.21.15.5:0/3783637227 shutdown_connections 2024-03-18T15:34:46.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.680+0000 7f2962285640 1 -- 172.21.15.5:0/3783637227 >> 172.21.15.5:0/3783637227 conn(0x7f295c0b6680 msgr2=0x7f295c06d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:46.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.680+0000 7f2962285640 1 -- 172.21.15.5:0/3783637227 shutdown_connections 2024-03-18T15:34:46.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.680+0000 7f2962285640 1 -- 172.21.15.5:0/3783637227 wait complete. 2024-03-18T15:34:46.702 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-03-18T15:34:46.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.824+0000 7f809ca88640 1 Processor -- start 2024-03-18T15:34:46.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.824+0000 7f809ca88640 1 -- start start 2024-03-18T15:34:46.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.824+0000 7f809ca88640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f809806d2d0 0x7f80980b23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:46.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.824+0000 7f809ca88640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f809813f260 0x7f809813f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:46.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.824+0000 7f809ca88640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80980b0ea0 con 0x7f80981453a0 2024-03-18T15:34:46.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.824+0000 7f809ca88640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f80980728c0 con 0x7f809806d2d0 2024-03-18T15:34:46.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.824+0000 7f809ca88640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f809806d6b0 con 0x7f809813f260 2024-03-18T15:34:46.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.824+0000 7f809659c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f809806d2d0 0x7f80980b23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:46.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.824+0000 7f809659c640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f809806d2d0 0x7f80980b23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34234/0 (socket says 172.21.15.5:34234) 2024-03-18T15:34:46.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.824+0000 7f809659c640 1 -- 172.21.15.5:0/285798765 learned_addr learned my addr 172.21.15.5:0/285798765 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:46.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.828+0000 7f8095d9b640 1 --2- 172.21.15.5:0/285798765 >> v2:172.21.15.80:3300/0 conn(0x7f809813f260 0x7f809813f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:46.830 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.828+0000 7f809659c640 1 -- 172.21.15.5:0/285798765 >> v2:172.21.15.80:3300/0 conn(0x7f809813f260 msgr2=0x7f809813f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:46.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.828+0000 7f809659c640 1 --2- 172.21.15.5:0/285798765 >> v2:172.21.15.80:3300/0 conn(0x7f809813f260 0x7f809813f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:46.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.828+0000 7f809659c640 1 -- 172.21.15.5:0/285798765 >> v1:172.21.15.80:6789/0 conn(0x7f80981453a0 legacy=0x7f80981457f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:46.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.828+0000 7f809659c640 1 -- 172.21.15.5:0/285798765 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80980b0bc0 con 0x7f809806d2d0 2024-03-18T15:34:46.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.828+0000 7f8095d9b640 1 --2- 172.21.15.5:0/285798765 >> v2:172.21.15.80:3300/0 conn(0x7f809813f260 0x7f809813f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:46.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.828+0000 7f809659c640 1 --2- 172.21.15.5:0/285798765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f809806d2d0 0x7f80980b23a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f809809eaa0 tx=0x7f808801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67450b6e2ed0086 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:46.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.828+0000 7f809559a640 1 -- 172.21.15.5:0/285798765 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80880b0070 con 0x7f809806d2d0 2024-03-18T15:34:46.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.828+0000 7f809559a640 1 -- 172.21.15.5:0/285798765 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80880ab600 con 0x7f809806d2d0 2024-03-18T15:34:46.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.828+0000 7f809559a640 1 -- 172.21.15.5:0/285798765 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80880ab9d0 con 0x7f809806d2d0 2024-03-18T15:34:46.832 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.828+0000 7f809ca88640 1 -- 172.21.15.5:0/285798765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f809806d2d0 msgr2=0x7f80980b23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:46.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.828+0000 7f809ca88640 1 --2- 172.21.15.5:0/285798765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f809806d2d0 0x7f80980b23a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f809809eaa0 tx=0x7f808801c450 comp rx=0 tx=0).stop 2024-03-18T15:34:46.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.828+0000 7f809ca88640 1 -- 172.21.15.5:0/285798765 shutdown_connections 2024-03-18T15:34:46.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.828+0000 7f809ca88640 1 --2- 172.21.15.5:0/285798765 >> v2:172.21.15.80:3300/0 conn(0x7f809813f260 0x7f809813f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:46.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.828+0000 7f809ca88640 1 --2- 172.21.15.5:0/285798765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f809806d2d0 0x7f80980b23a0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:46.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.828+0000 7f809ca88640 1 -- 172.21.15.5:0/285798765 >> 172.21.15.5:0/285798765 conn(0x7f80980b5670 msgr2=0x7f80980b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:46.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.828+0000 7f809ca88640 1 -- 172.21.15.5:0/285798765 shutdown_connections 2024-03-18T15:34:46.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.828+0000 7f809ca88640 1 -- 172.21.15.5:0/285798765 wait complete. 2024-03-18T15:34:46.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.832+0000 7f809ca88640 1 Processor -- start 2024-03-18T15:34:46.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.832+0000 7f809ca88640 1 -- start start 2024-03-18T15:34:46.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.832+0000 7f809ca88640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f809806d2d0 0x7f809813e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:46.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.832+0000 7f809ca88640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f809813f260 0x7f8098070760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:46.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.832+0000 7f809ca88640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80981453a0 0x7f80980adab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:46.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.832+0000 7f809ca88640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8098072ed0 con 0x7f809806d2d0 2024-03-18T15:34:46.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.832+0000 7f809ca88640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8098072d50 con 0x7f80981453a0 2024-03-18T15:34:46.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.832+0000 7f809ca88640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f80980b5140 con 0x7f809813f260 2024-03-18T15:34:46.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.832+0000 7f809659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f809806d2d0 0x7f809813e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:46.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.832+0000 7f809659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f809806d2d0 0x7f809813e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34238/0 (socket says 172.21.15.5:34238) 2024-03-18T15:34:46.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.832+0000 7f809659c640 1 -- 172.21.15.5:0/285059074 learned_addr learned my addr 172.21.15.5:0/285059074 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:46.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.832+0000 7f809659c640 1 -- 172.21.15.5:0/285059074 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f809813f260 msgr2=0x7f8098070760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:46.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.832+0000 7f809659c640 1 --2- 172.21.15.5:0/285059074 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f809813f260 0x7f8098070760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:46.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.832+0000 7f809659c640 1 -- 172.21.15.5:0/285059074 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80981453a0 msgr2=0x7f80980adab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:46.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.832+0000 7f8096d9d640 1 --2- 172.21.15.5:0/285059074 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80981453a0 0x7f80980adab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:46.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.832+0000 7f809659c640 1 --2- 172.21.15.5:0/285059074 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80981453a0 0x7f80980adab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:46.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.832+0000 7f809659c640 1 -- 172.21.15.5:0/285059074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8098142e00 con 0x7f809806d2d0 2024-03-18T15:34:46.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.832+0000 7f8096d9d640 1 --2- 172.21.15.5:0/285059074 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80981453a0 0x7f80980adab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:46.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.832+0000 7f809659c640 1 --2- 172.21.15.5:0/285059074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f809806d2d0 0x7f809813e570 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f808801ab00 tx=0x7f80880191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:46.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.832+0000 7f808effd640 1 -- 172.21.15.5:0/285059074 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80880b0040 con 0x7f809806d2d0 2024-03-18T15:34:46.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.832+0000 7f808effd640 1 -- 172.21.15.5:0/285059074 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80880b4640 con 0x7f809806d2d0 2024-03-18T15:34:46.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.832+0000 7f809ca88640 1 -- 172.21.15.5:0/285059074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8098071af0 con 0x7f809806d2d0 2024-03-18T15:34:46.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.832+0000 7f808effd640 1 -- 172.21.15.5:0/285059074 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80880b4a30 con 0x7f809806d2d0 2024-03-18T15:34:46.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.832+0000 7f809ca88640 1 -- 172.21.15.5:0/285059074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80980af8c0 con 0x7f809806d2d0 2024-03-18T15:34:46.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.832+0000 7f809ca88640 1 -- 172.21.15.5:0/285059074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f809804f550 con 0x7f809806d2d0 2024-03-18T15:34:46.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.832+0000 7f808effd640 1 -- 172.21.15.5:0/285059074 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f80880ab4c0 con 0x7f809806d2d0 2024-03-18T15:34:46.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.832+0000 7f808effd640 1 --2- 172.21.15.5:0/285059074 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f8078067690 0x7f8078069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:46.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.832+0000 7f808effd640 1 -- 172.21.15.5:0/285059074 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7f808811f0a0 con 0x7f809806d2d0 2024-03-18T15:34:46.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.832+0000 7f8095d9b640 1 --2- 172.21.15.5:0/285059074 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f8078067690 0x7f8078069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:46.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.836+0000 7f8095d9b640 1 --2- 172.21.15.5:0/285059074 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f8078067690 0x7f8078069b50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f80980aebd0 tx=0x7f8080008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:46.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.840+0000 7f808effd640 1 -- 172.21.15.5:0/285059074 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f80880160d0 con 0x7f809806d2d0 2024-03-18T15:34:46.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.992+0000 7f809ca88640 1 -- 172.21.15.5:0/285059074 --> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f8098143790 con 0x7f8078067690 2024-03-18T15:34:46.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.996+0000 7f808effd640 1 -- 172.21.15.5:0/285059074 <== mgr.4110 v2:172.21.15.5:6832/3362034860 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f8098143790 con 0x7f8078067690 2024-03-18T15:34:46.998 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 5 to deep-scrub 2024-03-18T15:34:47.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.996+0000 7f809ca88640 1 -- 172.21.15.5:0/285059074 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f8078067690 msgr2=0x7f8078069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:47.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.996+0000 7f809ca88640 1 --2- 172.21.15.5:0/285059074 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f8078067690 0x7f8078069b50 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f80980aebd0 tx=0x7f8080008040 comp rx=0 tx=0).stop 2024-03-18T15:34:47.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.996+0000 7f809ca88640 1 -- 172.21.15.5:0/285059074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f809806d2d0 msgr2=0x7f809813e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:47.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.996+0000 7f809ca88640 1 --2- 172.21.15.5:0/285059074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f809806d2d0 0x7f809813e570 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f808801ab00 tx=0x7f80880191f0 comp rx=0 tx=0).stop 2024-03-18T15:34:47.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.996+0000 7f809ca88640 1 -- 172.21.15.5:0/285059074 shutdown_connections 2024-03-18T15:34:47.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.996+0000 7f809ca88640 1 --2- 172.21.15.5:0/285059074 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f8078067690 0x7f8078069b50 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:47.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.996+0000 7f809ca88640 1 --2- 172.21.15.5:0/285059074 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80981453a0 0x7f80980adab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:47.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.996+0000 7f809ca88640 1 --2- 172.21.15.5:0/285059074 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f809813f260 0x7f8098070760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:47.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.996+0000 7f809ca88640 1 --2- 172.21.15.5:0/285059074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f809806d2d0 0x7f809813e570 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:47.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.996+0000 7f809ca88640 1 -- 172.21.15.5:0/285059074 >> 172.21.15.5:0/285059074 conn(0x7f80980b5670 msgr2=0x7f80980b5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:47.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.996+0000 7f809ca88640 1 -- 172.21.15.5:0/285059074 shutdown_connections 2024-03-18T15:34:47.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:46.996+0000 7f809ca88640 1 -- 172.21.15.5:0/285059074 wait complete. 2024-03-18T15:34:47.015 INFO:tasks.ceph:Scrubbing osd.6 2024-03-18T15:34:47.015 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-03-18T15:34:47.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.140+0000 7f3c1bada640 1 Processor -- start 2024-03-18T15:34:47.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.140+0000 7f3c1bada640 1 -- start start 2024-03-18T15:34:47.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.140+0000 7f3c1bada640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3c1406d300 0x7f3c140aea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:47.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.140+0000 7f3c1bada640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3c140ad8e0 0x7f3c1414ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:47.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.140+0000 7f3c1bada640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c1414b4f0 con 0x7f3c1414b060 2024-03-18T15:34:47.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.140+0000 7f3c1bada640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f3c140b5370 con 0x7f3c140ad8e0 2024-03-18T15:34:47.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.140+0000 7f3c1bada640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f3c140ae150 con 0x7f3c1406d300 2024-03-18T15:34:47.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.140+0000 7f3c19876640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3c1406d300 0x7f3c140aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:47.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.140+0000 7f3c19876640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3c1406d300 0x7f3c140aea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:56030/0 (socket says 172.21.15.5:56030) 2024-03-18T15:34:47.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.140+0000 7f3c19876640 1 -- 172.21.15.5:0/3455047111 learned_addr learned my addr 172.21.15.5:0/3455047111 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:47.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.140+0000 7f3c19075640 1 --2- 172.21.15.5:0/3455047111 >> v2:172.21.15.80:3300/0 conn(0x7f3c140ad8e0 0x7f3c1414ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:47.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.140+0000 7f3c19876640 1 -- 172.21.15.5:0/3455047111 >> v2:172.21.15.80:3300/0 conn(0x7f3c140ad8e0 msgr2=0x7f3c1414ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:47.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.140+0000 7f3c19876640 1 --2- 172.21.15.5:0/3455047111 >> v2:172.21.15.80:3300/0 conn(0x7f3c140ad8e0 0x7f3c1414ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:47.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.140+0000 7f3c19876640 1 -- 172.21.15.5:0/3455047111 >> v1:172.21.15.80:6789/0 conn(0x7f3c1414b060 legacy=0x7f3c14072490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:47.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.140+0000 7f3c19876640 1 -- 172.21.15.5:0/3455047111 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c1406d6e0 con 0x7f3c1406d300 2024-03-18T15:34:47.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.144+0000 7f3c19876640 1 --2- 172.21.15.5:0/3455047111 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3c1406d300 0x7f3c140aea60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3c08009870 tx=0x7f3c0801c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35cfeb757397eaf6 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:47.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.144+0000 7f3c18874640 1 -- 172.21.15.5:0/3455047111 <== mon.2 v2:172.21.15.80:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c080b1070 con 0x7f3c1406d300 2024-03-18T15:34:47.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.144+0000 7f3c18874640 1 -- 172.21.15.5:0/3455047111 <== mon.2 v2:172.21.15.80:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c080a8050 con 0x7f3c1406d300 2024-03-18T15:34:47.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.144+0000 7f3c18874640 1 -- 172.21.15.5:0/3455047111 <== mon.2 v2:172.21.15.80:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c080ac860 con 0x7f3c1406d300 2024-03-18T15:34:47.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.144+0000 7f3c1bada640 1 -- 172.21.15.5:0/3455047111 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3c1406d300 msgr2=0x7f3c140aea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:47.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.144+0000 7f3c1bada640 1 --2- 172.21.15.5:0/3455047111 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3c1406d300 0x7f3c140aea60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3c08009870 tx=0x7f3c0801c4d0 comp rx=0 tx=0).stop 2024-03-18T15:34:47.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.144+0000 7f3c1bada640 1 -- 172.21.15.5:0/3455047111 shutdown_connections 2024-03-18T15:34:47.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.144+0000 7f3c1bada640 1 --2- 172.21.15.5:0/3455047111 >> v2:172.21.15.80:3300/0 conn(0x7f3c140ad8e0 0x7f3c1414ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:47.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.144+0000 7f3c1bada640 1 --2- 172.21.15.5:0/3455047111 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3c1406d300 0x7f3c140aea60 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:47.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.144+0000 7f3c1bada640 1 -- 172.21.15.5:0/3455047111 >> 172.21.15.5:0/3455047111 conn(0x7f3c140b5670 msgr2=0x7f3c140b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:47.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.144+0000 7f3c1bada640 1 -- 172.21.15.5:0/3455047111 shutdown_connections 2024-03-18T15:34:47.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.144+0000 7f3c1bada640 1 -- 172.21.15.5:0/3455047111 wait complete. 2024-03-18T15:34:47.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.148+0000 7f3c1bada640 1 Processor -- start 2024-03-18T15:34:47.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.148+0000 7f3c1bada640 1 -- start start 2024-03-18T15:34:47.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.148+0000 7f3c1bada640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c1406d300 0x7f3c14145490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:47.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.148+0000 7f3c1bada640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c140ad8e0 0x7f3c141460e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:47.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.148+0000 7f3c1bada640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3c1414b060 0x7f3c14146620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:47.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.148+0000 7f3c1bada640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3c141548f0 con 0x7f3c140ad8e0 2024-03-18T15:34:47.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.148+0000 7f3c1bada640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3c14154770 con 0x7f3c1406d300 2024-03-18T15:34:47.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.148+0000 7f3c1bada640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f3c14154a70 con 0x7f3c1414b060 2024-03-18T15:34:47.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.148+0000 7f3c19075640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c140ad8e0 0x7f3c141460e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:47.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.148+0000 7f3c19075640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c140ad8e0 0x7f3c141460e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34242/0 (socket says 172.21.15.5:34242) 2024-03-18T15:34:47.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.148+0000 7f3c19075640 1 -- 172.21.15.5:0/4236001255 learned_addr learned my addr 172.21.15.5:0/4236001255 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:47.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.148+0000 7f3c19876640 1 --2- 172.21.15.5:0/4236001255 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c1406d300 0x7f3c14145490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:47.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.148+0000 7f3c1a077640 1 --2- 172.21.15.5:0/4236001255 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3c1414b060 0x7f3c14146620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:47.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.148+0000 7f3c19075640 1 -- 172.21.15.5:0/4236001255 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3c1414b060 msgr2=0x7f3c14146620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:47.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.148+0000 7f3c19075640 1 --2- 172.21.15.5:0/4236001255 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3c1414b060 0x7f3c14146620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:47.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.148+0000 7f3c19075640 1 -- 172.21.15.5:0/4236001255 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c1406d300 msgr2=0x7f3c14145490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:47.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.148+0000 7f3c19075640 1 --2- 172.21.15.5:0/4236001255 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c1406d300 0x7f3c14145490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:47.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.148+0000 7f3c19075640 1 -- 172.21.15.5:0/4236001255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c141e1490 con 0x7f3c140ad8e0 2024-03-18T15:34:47.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.148+0000 7f3c1a077640 1 --2- 172.21.15.5:0/4236001255 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f3c1414b060 0x7f3c14146620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:47.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.148+0000 7f3c19075640 1 --2- 172.21.15.5:0/4236001255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c140ad8e0 0x7f3c141460e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3c1000efc0 tx=0x7f3c1000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:47.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.148+0000 7f3c067fc640 1 -- 172.21.15.5:0/4236001255 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c10018020 con 0x7f3c140ad8e0 2024-03-18T15:34:47.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.148+0000 7f3c067fc640 1 -- 172.21.15.5:0/4236001255 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c10014480 con 0x7f3c140ad8e0 2024-03-18T15:34:47.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.148+0000 7f3c1bada640 1 -- 172.21.15.5:0/4236001255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c141e0e40 con 0x7f3c140ad8e0 2024-03-18T15:34:47.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.148+0000 7f3c067fc640 1 -- 172.21.15.5:0/4236001255 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c10014850 con 0x7f3c140ad8e0 2024-03-18T15:34:47.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.148+0000 7f3c1bada640 1 -- 172.21.15.5:0/4236001255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c140b2480 con 0x7f3c140ad8e0 2024-03-18T15:34:47.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.148+0000 7f3c1bada640 1 -- 172.21.15.5:0/4236001255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3c1404f5c0 con 0x7f3c140ad8e0 2024-03-18T15:34:47.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.152+0000 7f3c067fc640 1 -- 172.21.15.5:0/4236001255 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f3c1001b020 con 0x7f3c140ad8e0 2024-03-18T15:34:47.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.152+0000 7f3c067fc640 1 --2- 172.21.15.5:0/4236001255 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3bf80676b0 0x7f3bf8069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:47.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.152+0000 7f3c067fc640 1 -- 172.21.15.5:0/4236001255 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7f3c10086bd0 con 0x7f3c140ad8e0 2024-03-18T15:34:47.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.152+0000 7f3c19876640 1 --2- 172.21.15.5:0/4236001255 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3bf80676b0 0x7f3bf8069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:47.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.152+0000 7f3c067fc640 1 --2- 172.21.15.5:0/4236001255 >> [v2:172.21.15.80:6803/488342527,v1:172.21.15.80:6805/488342527] conn(0x7f3bf806d570 0x7f3bf806f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:47.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.152+0000 7f3c067fc640 1 -- 172.21.15.5:0/4236001255 --> [v2:172.21.15.80:6803/488342527,v1:172.21.15.80:6805/488342527] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3c100091e0 con 0x7f3bf806d570 2024-03-18T15:34:47.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.152+0000 7f3c067fc640 1 -- 172.21.15.5:0/4236001255 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f3c100870b0 con 0x7f3c140ad8e0 2024-03-18T15:34:47.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.152+0000 7f3c19876640 1 --2- 172.21.15.5:0/4236001255 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3bf80676b0 0x7f3bf8069b70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3c08009870 tx=0x7f3c080a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:47.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.152+0000 7f3c1a077640 1 --2- 172.21.15.5:0/4236001255 >> [v2:172.21.15.80:6803/488342527,v1:172.21.15.80:6805/488342527] conn(0x7f3bf806d570 0x7f3bf806f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:47.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.152+0000 7f3c1a077640 1 --2- 172.21.15.5:0/4236001255 >> [v2:172.21.15.80:6803/488342527,v1:172.21.15.80:6805/488342527] conn(0x7f3bf806d570 0x7f3bf806f990 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:47.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.152+0000 7f3c067fc640 1 -- 172.21.15.5:0/4236001255 <== osd.6 v2:172.21.15.80:6803/488342527 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3c100091e0 con 0x7f3bf806d570 2024-03-18T15:34:47.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.168+0000 7f3c1bada640 1 -- 172.21.15.5:0/4236001255 --> [v2:172.21.15.80:6803/488342527,v1:172.21.15.80:6805/488342527] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f3c141e0c60 con 0x7f3bf806d570 2024-03-18T15:34:47.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.168+0000 7f3c067fc640 1 -- 172.21.15.5:0/4236001255 <== osd.6 v2:172.21.15.80:6803/488342527 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f3c141e0c60 con 0x7f3bf806d570 2024-03-18T15:34:47.171 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-03-18T15:34:47.171 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-18T15:34:47.171 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-03-18T15:34:47.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.168+0000 7f3c1bada640 1 -- 172.21.15.5:0/4236001255 >> [v2:172.21.15.80:6803/488342527,v1:172.21.15.80:6805/488342527] conn(0x7f3bf806d570 msgr2=0x7f3bf806f990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:47.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.168+0000 7f3c1bada640 1 --2- 172.21.15.5:0/4236001255 >> [v2:172.21.15.80:6803/488342527,v1:172.21.15.80:6805/488342527] conn(0x7f3bf806d570 0x7f3bf806f990 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:47.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.168+0000 7f3c1bada640 1 -- 172.21.15.5:0/4236001255 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3bf80676b0 msgr2=0x7f3bf8069b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:47.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.168+0000 7f3c1bada640 1 --2- 172.21.15.5:0/4236001255 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3bf80676b0 0x7f3bf8069b70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3c08009870 tx=0x7f3c080a7000 comp rx=0 tx=0).stop 2024-03-18T15:34:47.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.168+0000 7f3c1bada640 1 -- 172.21.15.5:0/4236001255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c140ad8e0 msgr2=0x7f3c141460e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:47.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.168+0000 7f3c1bada640 1 --2- 172.21.15.5:0/4236001255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c140ad8e0 0x7f3c141460e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3c1000efc0 tx=0x7f3c1000c490 comp rx=0 tx=0).stop 2024-03-18T15:34:47.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.168+0000 7f3c1a077640 1 -- 172.21.15.5:0/4236001255 reap_dead start 2024-03-18T15:34:47.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.168+0000 7f3c1bada640 1 -- 172.21.15.5:0/4236001255 shutdown_connections 2024-03-18T15:34:47.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.168+0000 7f3c1bada640 1 -- 172.21.15.5:0/4236001255 >> 172.21.15.5:0/4236001255 conn(0x7f3c140b5670 msgr2=0x7f3c1413dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:47.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.168+0000 7f3c1bada640 1 -- 172.21.15.5:0/4236001255 shutdown_connections 2024-03-18T15:34:47.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.168+0000 7f3c1bada640 1 -- 172.21.15.5:0/4236001255 wait complete. 2024-03-18T15:34:47.187 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-03-18T15:34:47.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.304+0000 7f84e8f02640 1 Processor -- start 2024-03-18T15:34:47.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.304+0000 7f84e8f02640 1 -- start start 2024-03-18T15:34:47.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.304+0000 7f84e8f02640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84e40af790 0x7f84e4072b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:47.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.304+0000 7f84e8f02640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f84e40b26a0 0x7f84e40b2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:47.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.304+0000 7f84e8f02640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84e40b3070 con 0x7f84e406d070 2024-03-18T15:34:47.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.304+0000 7f84e8f02640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f84e40afb70 con 0x7f84e40b26a0 2024-03-18T15:34:47.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.304+0000 7f84e8f02640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f84e40b4a20 con 0x7f84e40af790 2024-03-18T15:34:47.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.304+0000 7f84e2d9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f84e40b26a0 0x7f84e40b2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:47.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.304+0000 7f84e2d9d640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f84e40b26a0 0x7f84e40b2b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34244/0 (socket says 172.21.15.5:34244) 2024-03-18T15:34:47.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.304+0000 7f84e2d9d640 1 -- 172.21.15.5:0/3716598786 learned_addr learned my addr 172.21.15.5:0/3716598786 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:47.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.304+0000 7f84e259c640 1 --2- 172.21.15.5:0/3716598786 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84e40af790 0x7f84e4072b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:47.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.304+0000 7f84e2d9d640 1 -- 172.21.15.5:0/3716598786 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84e40af790 msgr2=0x7f84e4072b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:47.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.304+0000 7f84e2d9d640 1 --2- 172.21.15.5:0/3716598786 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84e40af790 0x7f84e4072b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:47.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.304+0000 7f84e2d9d640 1 -- 172.21.15.5:0/3716598786 >> v1:172.21.15.80:6789/0 conn(0x7f84e406d070 legacy=0x7f84e4147780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:47.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.304+0000 7f84e2d9d640 1 -- 172.21.15.5:0/3716598786 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e40b4d00 con 0x7f84e40b26a0 2024-03-18T15:34:47.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.304+0000 7f84e259c640 1 --2- 172.21.15.5:0/3716598786 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84e40af790 0x7f84e4072b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:47.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.308+0000 7f84e2d9d640 1 --2- 172.21.15.5:0/3716598786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84e40b26a0 0x7f84e40b2b30 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f84dc009e50 tx=0x7f84dc01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e89b600ebf2d8eb server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:47.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.308+0000 7f84e159a640 1 -- 172.21.15.5:0/3716598786 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84dc0a55f0 con 0x7f84e40b26a0 2024-03-18T15:34:47.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.308+0000 7f84e159a640 1 -- 172.21.15.5:0/3716598786 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84dc0a5790 con 0x7f84e40b26a0 2024-03-18T15:34:47.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.308+0000 7f84e159a640 1 -- 172.21.15.5:0/3716598786 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84dc0a5b60 con 0x7f84e40b26a0 2024-03-18T15:34:47.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.308+0000 7f84e8f02640 1 -- 172.21.15.5:0/3716598786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84e40b26a0 msgr2=0x7f84e40b2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:47.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.308+0000 7f84e8f02640 1 --2- 172.21.15.5:0/3716598786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84e40b26a0 0x7f84e40b2b30 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f84dc009e50 tx=0x7f84dc01c960 comp rx=0 tx=0).stop 2024-03-18T15:34:47.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.308+0000 7f84e8f02640 1 -- 172.21.15.5:0/3716598786 shutdown_connections 2024-03-18T15:34:47.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.308+0000 7f84e8f02640 1 --2- 172.21.15.5:0/3716598786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84e40b26a0 0x7f84e40b2b30 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:47.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.308+0000 7f84e8f02640 1 --2- 172.21.15.5:0/3716598786 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84e40af790 0x7f84e4072b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:47.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.308+0000 7f84e8f02640 1 -- 172.21.15.5:0/3716598786 >> 172.21.15.5:0/3716598786 conn(0x7f84e40b5670 msgr2=0x7f84e40b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:47.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.308+0000 7f84e8f02640 1 -- 172.21.15.5:0/3716598786 shutdown_connections 2024-03-18T15:34:47.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.308+0000 7f84e8f02640 1 -- 172.21.15.5:0/3716598786 wait complete. 2024-03-18T15:34:47.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.308+0000 7f84e8f02640 1 Processor -- start 2024-03-18T15:34:47.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.308+0000 7f84e8f02640 1 -- start start 2024-03-18T15:34:47.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.308+0000 7f84e8f02640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84e406d070 0x7f84e4070cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:47.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.308+0000 7f84e8f02640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84e40af790 0x7f84e41ddad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:47.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.308+0000 7f84e8f02640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84e40b26a0 0x7f84e406f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:47.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.308+0000 7f84e8f02640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f84e40b3900 con 0x7f84e40af790 2024-03-18T15:34:47.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.308+0000 7f84e8f02640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f84e40b3780 con 0x7f84e40b26a0 2024-03-18T15:34:47.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.308+0000 7f84e8f02640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f84e40b3a80 con 0x7f84e406d070 2024-03-18T15:34:47.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.308+0000 7f84e1d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84e40af790 0x7f84e41ddad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:47.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.308+0000 7f84e1d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84e40af790 0x7f84e41ddad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34258/0 (socket says 172.21.15.5:34258) 2024-03-18T15:34:47.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.308+0000 7f84e1d9b640 1 -- 172.21.15.5:0/2839868000 learned_addr learned my addr 172.21.15.5:0/2839868000 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:47.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.308+0000 7f84e2d9d640 1 --2- 172.21.15.5:0/2839868000 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84e40b26a0 0x7f84e406f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:47.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.308+0000 7f84e259c640 1 --2- 172.21.15.5:0/2839868000 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84e406d070 0x7f84e4070cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:47.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.308+0000 7f84e1d9b640 1 -- 172.21.15.5:0/2839868000 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84e406d070 msgr2=0x7f84e4070cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:47.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.308+0000 7f84e1d9b640 1 --2- 172.21.15.5:0/2839868000 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84e406d070 0x7f84e4070cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:47.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.308+0000 7f84e1d9b640 1 -- 172.21.15.5:0/2839868000 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84e40b26a0 msgr2=0x7f84e406f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:47.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.308+0000 7f84e1d9b640 1 --2- 172.21.15.5:0/2839868000 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84e40b26a0 0x7f84e406f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:47.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.308+0000 7f84e1d9b640 1 -- 172.21.15.5:0/2839868000 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e406ff20 con 0x7f84e40af790 2024-03-18T15:34:47.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.308+0000 7f84e1d9b640 1 --2- 172.21.15.5:0/2839868000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84e40af790 0x7f84e41ddad0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f84d800c450 tx=0x7f84d800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:47.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.312+0000 7f84d2ffd640 1 -- 172.21.15.5:0/2839868000 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d8012830 con 0x7f84e40af790 2024-03-18T15:34:47.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.312+0000 7f84d2ffd640 1 -- 172.21.15.5:0/2839868000 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84d8017070 con 0x7f84e40af790 2024-03-18T15:34:47.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.312+0000 7f84e8f02640 1 -- 172.21.15.5:0/2839868000 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84e4072170 con 0x7f84e40af790 2024-03-18T15:34:47.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.312+0000 7f84d2ffd640 1 -- 172.21.15.5:0/2839868000 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d8012c30 con 0x7f84e40af790 2024-03-18T15:34:47.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.312+0000 7f84e8f02640 1 -- 172.21.15.5:0/2839868000 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84e406fa80 con 0x7f84e40af790 2024-03-18T15:34:47.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.312+0000 7f84e8f02640 1 -- 172.21.15.5:0/2839868000 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84e404f550 con 0x7f84e40af790 2024-03-18T15:34:47.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.312+0000 7f84d2ffd640 1 -- 172.21.15.5:0/2839868000 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f84d801c460 con 0x7f84e40af790 2024-03-18T15:34:47.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.312+0000 7f84d2ffd640 1 --2- 172.21.15.5:0/2839868000 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f84c4067660 0x7f84c4069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:47.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.312+0000 7f84e259c640 1 --2- 172.21.15.5:0/2839868000 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f84c4067660 0x7f84c4069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:47.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.312+0000 7f84d2ffd640 1 -- 172.21.15.5:0/2839868000 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7f84d8085320 con 0x7f84e40af790 2024-03-18T15:34:47.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.312+0000 7f84e259c640 1 --2- 172.21.15.5:0/2839868000 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f84c4067660 0x7f84c4069b20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f84e4072d20 tx=0x7f84d4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:47.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.316+0000 7f84d2ffd640 1 -- 172.21.15.5:0/2839868000 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f84d80562a0 con 0x7f84e40af790 2024-03-18T15:34:47.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.476+0000 7f84e8f02640 1 -- 172.21.15.5:0/2839868000 --> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f84e4148110 con 0x7f84c4067660 2024-03-18T15:34:47.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.476+0000 7f84d2ffd640 1 -- 172.21.15.5:0/2839868000 <== mgr.4110 v2:172.21.15.5:6832/3362034860 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f84e4148110 con 0x7f84c4067660 2024-03-18T15:34:47.479 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 6 to deep-scrub 2024-03-18T15:34:47.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.480+0000 7f84e8f02640 1 -- 172.21.15.5:0/2839868000 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f84c4067660 msgr2=0x7f84c4069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:47.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.480+0000 7f84e8f02640 1 --2- 172.21.15.5:0/2839868000 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f84c4067660 0x7f84c4069b20 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f84e4072d20 tx=0x7f84d4008040 comp rx=0 tx=0).stop 2024-03-18T15:34:47.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.480+0000 7f84e8f02640 1 -- 172.21.15.5:0/2839868000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84e40af790 msgr2=0x7f84e41ddad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:47.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.480+0000 7f84e8f02640 1 --2- 172.21.15.5:0/2839868000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84e40af790 0x7f84e41ddad0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f84d800c450 tx=0x7f84d800c910 comp rx=0 tx=0).stop 2024-03-18T15:34:47.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.480+0000 7f84e8f02640 1 -- 172.21.15.5:0/2839868000 shutdown_connections 2024-03-18T15:34:47.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.480+0000 7f84e8f02640 1 --2- 172.21.15.5:0/2839868000 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f84c4067660 0x7f84c4069b20 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:47.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.480+0000 7f84e8f02640 1 --2- 172.21.15.5:0/2839868000 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84e40b26a0 0x7f84e406f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:47.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.480+0000 7f84e8f02640 1 --2- 172.21.15.5:0/2839868000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84e40af790 0x7f84e41ddad0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:47.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.480+0000 7f84e8f02640 1 --2- 172.21.15.5:0/2839868000 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f84e406d070 0x7f84e4070cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:47.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.480+0000 7f84e8f02640 1 -- 172.21.15.5:0/2839868000 >> 172.21.15.5:0/2839868000 conn(0x7f84e40b5670 msgr2=0x7f84e40b80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:47.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.480+0000 7f84e8f02640 1 -- 172.21.15.5:0/2839868000 shutdown_connections 2024-03-18T15:34:47.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.480+0000 7f84e8f02640 1 -- 172.21.15.5:0/2839868000 wait complete. 2024-03-18T15:34:47.498 INFO:tasks.ceph:Scrubbing osd.7 2024-03-18T15:34:47.499 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-03-18T15:34:47.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.624+0000 7f7616d8c640 1 Processor -- start 2024-03-18T15:34:47.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.624+0000 7f7616d8c640 1 -- start start 2024-03-18T15:34:47.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.624+0000 7f7616d8c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f761006d300 0x7f7610145250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:47.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.624+0000 7f7616d8c640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76100afd60 con 0x7f76100b4820 2024-03-18T15:34:47.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.624+0000 7f7616d8c640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76100b52b0 con 0x7f761006f640 2024-03-18T15:34:47.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.624+0000 7f7616d8c640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f761006d6e0 con 0x7f761006d300 2024-03-18T15:34:47.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.624+0000 7f760ffff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f76100b4820 0x7f761013efb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:39918/0 (socket says 172.21.15.5:39918) 2024-03-18T15:34:47.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.624+0000 7f760ffff640 1 -- 172.21.15.5:0/4146779850 learned_addr learned my addr 172.21.15.5:0/4146779850 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:34:47.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.624+0000 7f7614b28640 1 --2- 172.21.15.5:0/4146779850 >> v2:172.21.15.80:3300/0 conn(0x7f761006d300 0x7f7610145250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:47.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.624+0000 7f760f7fe640 1 -- 172.21.15.5:0/4146779850 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3718167121 0 0) 0x7f76100afd60 con 0x7f76100b4820 2024-03-18T15:34:47.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.624+0000 7f760f7fe640 1 -- 172.21.15.5:0/4146779850 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f75fc003530 con 0x7f76100b4820 2024-03-18T15:34:47.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.624+0000 7f760f7fe640 1 -- 172.21.15.5:0/4146779850 <== mon.0 v1:172.21.15.5:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 286373251 0 0) 0x7f75fc003530 con 0x7f76100b4820 2024-03-18T15:34:47.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.624+0000 7f760f7fe640 1 -- 172.21.15.5:0/4146779850 >> v2:172.21.15.80:3300/0 conn(0x7f761006d300 msgr2=0x7f7610145250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:47.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.624+0000 7f760f7fe640 1 --2- 172.21.15.5:0/4146779850 >> v2:172.21.15.80:3300/0 conn(0x7f761006d300 0x7f7610145250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:47.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.624+0000 7f760f7fe640 1 -- 172.21.15.5:0/4146779850 >> v1:172.21.15.80:6789/0 conn(0x7f761006f640 legacy=0x7f761014fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:47.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.624+0000 7f760f7fe640 1 -- 172.21.15.5:0/4146779850 --> v1:172.21.15.5:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7610070960 con 0x7f76100b4820 2024-03-18T15:34:47.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.624+0000 7f760f7fe640 1 -- 172.21.15.5:0/4146779850 <== mon.0 v1:172.21.15.5:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3837549003 0 0) 0x7f75f8001680 con 0x7f76100b4820 2024-03-18T15:34:47.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.624+0000 7f760f7fe640 1 -- 172.21.15.5:0/4146779850 >> v1:172.21.15.5:6789/0 conn(0x7f76100b4820 legacy=0x7f761013efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:47.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.624+0000 7f760f7fe640 1 --2- 172.21.15.5:0/4146779850 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f75fc003aa0 0x7f75fc003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:47.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.624+0000 7f760f7fe640 1 --2- 172.21.15.5:0/4146779850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75fc004810 0x7f75fc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:47.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.624+0000 7f760f7fe640 1 --2- 172.21.15.5:0/4146779850 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f75fc0053d0 0x7f75fc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:47.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.624+0000 7f760f7fe640 1 -- 172.21.15.5:0/4146779850 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f761006d6e0 con 0x7f75fc004810 2024-03-18T15:34:47.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.624+0000 7f760f7fe640 1 -- 172.21.15.5:0/4146779850 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f75fc00c290 con 0x7f75fc0053d0 2024-03-18T15:34:47.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.624+0000 7f760f7fe640 1 -- 172.21.15.5:0/4146779850 --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f75fc00c490 con 0x7f75fc003aa0 2024-03-18T15:34:47.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.624+0000 7f760ffff640 1 --2- 172.21.15.5:0/4146779850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75fc004810 0x7f75fc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:47.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.628+0000 7f7615329640 1 --2- 172.21.15.5:0/4146779850 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f75fc0053d0 0x7f75fc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:47.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.628+0000 7f7614b28640 1 --2- 172.21.15.5:0/4146779850 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f75fc003aa0 0x7f75fc003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:47.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.628+0000 7f760ffff640 1 -- 172.21.15.5:0/4146779850 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f75fc003aa0 msgr2=0x7f75fc003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:47.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.628+0000 7f760ffff640 1 --2- 172.21.15.5:0/4146779850 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f75fc003aa0 0x7f75fc003ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:47.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.628+0000 7f760ffff640 1 -- 172.21.15.5:0/4146779850 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f75fc0053d0 msgr2=0x7f75fc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:47.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.628+0000 7f760ffff640 1 --2- 172.21.15.5:0/4146779850 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f75fc0053d0 0x7f75fc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:47.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.628+0000 7f760ffff640 1 -- 172.21.15.5:0/4146779850 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75fc003530 con 0x7f75fc004810 2024-03-18T15:34:47.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.628+0000 7f7615329640 1 -- 172.21.15.5:0/4146779850 reap_dead start 2024-03-18T15:34:47.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.628+0000 7f760ffff640 1 --2- 172.21.15.5:0/4146779850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75fc004810 0x7f75fc004c90 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f75f800c210 tx=0x7f75f801cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c1ce86deb4cdee1 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:47.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.628+0000 7f760f7fe640 1 -- 172.21.15.5:0/4146779850 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f8007d20 con 0x7f75fc004810 2024-03-18T15:34:47.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.628+0000 7f760f7fe640 1 -- 172.21.15.5:0/4146779850 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75f80a9070 con 0x7f75fc004810 2024-03-18T15:34:47.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.628+0000 7f760f7fe640 1 -- 172.21.15.5:0/4146779850 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f80a5dd0 con 0x7f75fc004810 2024-03-18T15:34:47.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.628+0000 7f7616d8c640 1 -- 172.21.15.5:0/4146779850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75fc004810 msgr2=0x7f75fc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:47.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.628+0000 7f7616d8c640 1 --2- 172.21.15.5:0/4146779850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75fc004810 0x7f75fc004c90 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f75f800c210 tx=0x7f75f801cc30 comp rx=0 tx=0).stop 2024-03-18T15:34:47.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.628+0000 7f7616d8c640 1 -- 172.21.15.5:0/4146779850 shutdown_connections 2024-03-18T15:34:47.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.628+0000 7f7616d8c640 1 --2- 172.21.15.5:0/4146779850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75fc004810 0x7f75fc004c90 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:47.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.628+0000 7f7616d8c640 1 -- 172.21.15.5:0/4146779850 >> 172.21.15.5:0/4146779850 conn(0x7f76100b5670 msgr2=0x7f76100b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:47.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.628+0000 7f7616d8c640 1 -- 172.21.15.5:0/4146779850 shutdown_connections 2024-03-18T15:34:47.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.628+0000 7f7616d8c640 1 -- 172.21.15.5:0/4146779850 wait complete. 2024-03-18T15:34:47.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.628+0000 7f7616d8c640 1 Processor -- start 2024-03-18T15:34:47.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.628+0000 7f7616d8c640 1 -- start start 2024-03-18T15:34:47.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.632+0000 7f7616d8c640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f75fc004810 0x7f761013a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:47.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.632+0000 7f7616d8c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f761013ae40 0x7f76101de0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:47.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.632+0000 7f7616d8c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76101de5f0 0x7f7610138020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:47.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.632+0000 7f7616d8c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f76101457e0 con 0x7f76101de5f0 2024-03-18T15:34:47.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.632+0000 7f7616d8c640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7610145660 con 0x7f761013ae40 2024-03-18T15:34:47.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.632+0000 7f7616d8c640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f76100b3570 con 0x7f75fc004810 2024-03-18T15:34:47.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.632+0000 7f7615329640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76101de5f0 0x7f7610138020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:47.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.632+0000 7f7615329640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76101de5f0 0x7f7610138020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34276/0 (socket says 172.21.15.5:34276) 2024-03-18T15:34:47.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.632+0000 7f7615329640 1 -- 172.21.15.5:0/2929823959 learned_addr learned my addr 172.21.15.5:0/2929823959 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:47.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.632+0000 7f760ffff640 1 --2- 172.21.15.5:0/2929823959 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f761013ae40 0x7f76101de0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:47.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.632+0000 7f7614b28640 1 --2- 172.21.15.5:0/2929823959 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f75fc004810 0x7f761013a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:47.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.632+0000 7f7615329640 1 -- 172.21.15.5:0/2929823959 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f75fc004810 msgr2=0x7f761013a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:47.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.632+0000 7f7615329640 1 --2- 172.21.15.5:0/2929823959 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f75fc004810 0x7f761013a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:47.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.632+0000 7f7615329640 1 -- 172.21.15.5:0/2929823959 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f761013ae40 msgr2=0x7f76101de0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:47.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.632+0000 7f7615329640 1 --2- 172.21.15.5:0/2929823959 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f761013ae40 0x7f76101de0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:47.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.632+0000 7f7615329640 1 -- 172.21.15.5:0/2929823959 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f761013ffb0 con 0x7f76101de5f0 2024-03-18T15:34:47.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.632+0000 7f7614b28640 1 --2- 172.21.15.5:0/2929823959 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f75fc004810 0x7f761013a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:34:47.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.632+0000 7f7615329640 1 --2- 172.21.15.5:0/2929823959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76101de5f0 0x7f7610138020 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f75fc004790 tx=0x7f760000e610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:47.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.632+0000 7f760d7fa640 1 -- 172.21.15.5:0/2929823959 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7600019070 con 0x7f76101de5f0 2024-03-18T15:34:47.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.632+0000 7f760d7fa640 1 -- 172.21.15.5:0/2929823959 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76000050d0 con 0x7f76101de5f0 2024-03-18T15:34:47.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.632+0000 7f7616d8c640 1 -- 172.21.15.5:0/2929823959 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7610139450 con 0x7f76101de5f0 2024-03-18T15:34:47.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.632+0000 7f760d7fa640 1 -- 172.21.15.5:0/2929823959 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7600003600 con 0x7f76101de5f0 2024-03-18T15:34:47.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.632+0000 7f7616d8c640 1 -- 172.21.15.5:0/2929823959 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7610139a10 con 0x7f76101de5f0 2024-03-18T15:34:47.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.632+0000 7f7616d8c640 1 -- 172.21.15.5:0/2929823959 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f761004f5c0 con 0x7f76101de5f0 2024-03-18T15:34:47.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.632+0000 7f760d7fa640 1 -- 172.21.15.5:0/2929823959 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f760001c070 con 0x7f76101de5f0 2024-03-18T15:34:47.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.632+0000 7f760d7fa640 1 --2- 172.21.15.5:0/2929823959 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f75f0067660 0x7f75f0069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:47.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.632+0000 7f760d7fa640 1 -- 172.21.15.5:0/2929823959 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7f7600084f70 con 0x7f76101de5f0 2024-03-18T15:34:47.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.632+0000 7f7614b28640 1 --2- 172.21.15.5:0/2929823959 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f75f0067660 0x7f75f0069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:47.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.632+0000 7f760d7fa640 1 --2- 172.21.15.5:0/2929823959 >> [v2:172.21.15.80:6800/982831666,v1:172.21.15.80:6801/982831666] conn(0x7f75f006d520 0x7f75f006f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:47.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.632+0000 7f760d7fa640 1 -- 172.21.15.5:0/2929823959 --> [v2:172.21.15.80:6800/982831666,v1:172.21.15.80:6801/982831666] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f75f006fff0 con 0x7f75f006d520 2024-03-18T15:34:47.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.632+0000 7f760d7fa640 1 -- 172.21.15.5:0/2929823959 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7600085400 con 0x7f76101de5f0 2024-03-18T15:34:47.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.632+0000 7f7614b28640 1 --2- 172.21.15.5:0/2929823959 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f75f0067660 0x7f75f0069b20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7608007460 tx=0x7f760800b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:47.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.632+0000 7f760ffff640 1 --2- 172.21.15.5:0/2929823959 >> [v2:172.21.15.80:6800/982831666,v1:172.21.15.80:6801/982831666] conn(0x7f75f006d520 0x7f75f006f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:47.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.636+0000 7f760ffff640 1 --2- 172.21.15.5:0/2929823959 >> [v2:172.21.15.80:6800/982831666,v1:172.21.15.80:6801/982831666] conn(0x7f75f006d520 0x7f75f006f940 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:47.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.636+0000 7f760d7fa640 1 -- 172.21.15.5:0/2929823959 <== osd.7 v2:172.21.15.80:6800/982831666 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f75f006fff0 con 0x7f75f006d520 2024-03-18T15:34:47.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.648+0000 7f7616d8c640 1 -- 172.21.15.5:0/2929823959 --> [v2:172.21.15.80:6800/982831666,v1:172.21.15.80:6801/982831666] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f7610071e90 con 0x7f75f006d520 2024-03-18T15:34:47.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.652+0000 7f760d7fa640 1 -- 172.21.15.5:0/2929823959 <== osd.7 v2:172.21.15.80:6800/982831666 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f7610071e90 con 0x7f75f006d520 2024-03-18T15:34:47.653 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-03-18T15:34:47.653 INFO:teuthology.orchestra.run.smithi005.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-18T15:34:47.654 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-03-18T15:34:47.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.652+0000 7f7616d8c640 1 -- 172.21.15.5:0/2929823959 >> [v2:172.21.15.80:6800/982831666,v1:172.21.15.80:6801/982831666] conn(0x7f75f006d520 msgr2=0x7f75f006f940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:47.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.652+0000 7f7616d8c640 1 --2- 172.21.15.5:0/2929823959 >> [v2:172.21.15.80:6800/982831666,v1:172.21.15.80:6801/982831666] conn(0x7f75f006d520 0x7f75f006f940 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:47.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.652+0000 7f7616d8c640 1 -- 172.21.15.5:0/2929823959 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f75f0067660 msgr2=0x7f75f0069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:47.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.652+0000 7f7616d8c640 1 --2- 172.21.15.5:0/2929823959 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f75f0067660 0x7f75f0069b20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7608007460 tx=0x7f760800b040 comp rx=0 tx=0).stop 2024-03-18T15:34:47.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.652+0000 7f7616d8c640 1 -- 172.21.15.5:0/2929823959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76101de5f0 msgr2=0x7f7610138020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:47.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.652+0000 7f7616d8c640 1 --2- 172.21.15.5:0/2929823959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76101de5f0 0x7f7610138020 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f75fc004790 tx=0x7f760000e610 comp rx=0 tx=0).stop 2024-03-18T15:34:47.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.652+0000 7f7615329640 1 -- 172.21.15.5:0/2929823959 reap_dead start 2024-03-18T15:34:47.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.652+0000 7f7616d8c640 1 -- 172.21.15.5:0/2929823959 shutdown_connections 2024-03-18T15:34:47.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.652+0000 7f7616d8c640 1 -- 172.21.15.5:0/2929823959 >> 172.21.15.5:0/2929823959 conn(0x7f76100b5670 msgr2=0x7f76101d4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:47.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.652+0000 7f7616d8c640 1 -- 172.21.15.5:0/2929823959 shutdown_connections 2024-03-18T15:34:47.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.652+0000 7f7616d8c640 1 -- 172.21.15.5:0/2929823959 wait complete. 2024-03-18T15:34:47.670 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-03-18T15:34:47.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.784+0000 7f0fb2762640 1 Processor -- start 2024-03-18T15:34:47.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.784+0000 7f0fb2762640 1 -- start start 2024-03-18T15:34:47.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.784+0000 7f0fb2762640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0fac06d2d0 0x7f0fac141650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:47.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.784+0000 7f0fb2762640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0fac0ae0f0 0x7f0fac0b4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:47.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.784+0000 7f0fb2762640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0fac0ae9c0 0x7f0fac0b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:47.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.784+0000 7f0fb2762640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f0fac0724e0 con 0x7f0fac0ae9c0 2024-03-18T15:34:47.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.784+0000 7f0fb2762640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f0fac06d6b0 con 0x7f0fac0ae0f0 2024-03-18T15:34:47.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.784+0000 7f0fb2762640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f0fac072730 con 0x7f0fac06d2d0 2024-03-18T15:34:47.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.784+0000 7f0fb0cff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0fac0ae9c0 0x7f0fac0b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:47.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.784+0000 7f0fb0cff640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f0fac0ae9c0 0x7f0fac0b50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34288/0 (socket says 172.21.15.5:34288) 2024-03-18T15:34:47.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.784+0000 7f0fb0cff640 1 -- 172.21.15.5:0/3437166435 learned_addr learned my addr 172.21.15.5:0/3437166435 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:47.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.784+0000 7f0fabfff640 1 --2- 172.21.15.5:0/3437166435 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0fac06d2d0 0x7f0fac141650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:47.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.784+0000 7f0fab7fe640 1 --2- 172.21.15.5:0/3437166435 >> v2:172.21.15.80:3300/0 conn(0x7f0fac0ae0f0 0x7f0fac0b4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:47.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.784+0000 7f0fb0cff640 1 -- 172.21.15.5:0/3437166435 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0fac06d2d0 msgr2=0x7f0fac141650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:47.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.784+0000 7f0fb0cff640 1 --2- 172.21.15.5:0/3437166435 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0fac06d2d0 0x7f0fac141650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:47.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.784+0000 7f0fb0cff640 1 -- 172.21.15.5:0/3437166435 >> v2:172.21.15.80:3300/0 conn(0x7f0fac0ae0f0 msgr2=0x7f0fac0b4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:47.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.784+0000 7f0fb0cff640 1 --2- 172.21.15.5:0/3437166435 >> v2:172.21.15.80:3300/0 conn(0x7f0fac0ae0f0 0x7f0fac0b4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:47.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.784+0000 7f0fb0cff640 1 -- 172.21.15.5:0/3437166435 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fac072a10 con 0x7f0fac0ae9c0 2024-03-18T15:34:47.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.784+0000 7f0fab7fe640 1 --2- 172.21.15.5:0/3437166435 >> v2:172.21.15.80:3300/0 conn(0x7f0fac0ae0f0 0x7f0fac0b4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:34:47.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.788+0000 7f0fb0cff640 1 --2- 172.21.15.5:0/3437166435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fac0ae9c0 0x7f0fac0b50a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0fac072940 tx=0x7f0fa401c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d5355db9f07cbec server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:47.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.788+0000 7f0faaffd640 1 -- 172.21.15.5:0/3437166435 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa40a5550 con 0x7f0fac0ae9c0 2024-03-18T15:34:47.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.788+0000 7f0faaffd640 1 -- 172.21.15.5:0/3437166435 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fa40a56f0 con 0x7f0fac0ae9c0 2024-03-18T15:34:47.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.788+0000 7f0faaffd640 1 -- 172.21.15.5:0/3437166435 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa40a5ac0 con 0x7f0fac0ae9c0 2024-03-18T15:34:47.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.788+0000 7f0fb2762640 1 -- 172.21.15.5:0/3437166435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fac0ae9c0 msgr2=0x7f0fac0b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:47.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.788+0000 7f0fb2762640 1 --2- 172.21.15.5:0/3437166435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fac0ae9c0 0x7f0fac0b50a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0fac072940 tx=0x7f0fa401c8d0 comp rx=0 tx=0).stop 2024-03-18T15:34:47.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.788+0000 7f0fb2762640 1 -- 172.21.15.5:0/3437166435 shutdown_connections 2024-03-18T15:34:47.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.788+0000 7f0fb2762640 1 --2- 172.21.15.5:0/3437166435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fac0ae9c0 0x7f0fac0b50a0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:47.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.788+0000 7f0fb2762640 1 --2- 172.21.15.5:0/3437166435 >> v2:172.21.15.80:3300/0 conn(0x7f0fac0ae0f0 0x7f0fac0b4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:47.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.788+0000 7f0fb2762640 1 --2- 172.21.15.5:0/3437166435 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0fac06d2d0 0x7f0fac141650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:47.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.788+0000 7f0fb2762640 1 -- 172.21.15.5:0/3437166435 >> 172.21.15.5:0/3437166435 conn(0x7f0fac0b5670 msgr2=0x7f0fac0b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:47.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.788+0000 7f0fb2762640 1 -- 172.21.15.5:0/3437166435 shutdown_connections 2024-03-18T15:34:47.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.788+0000 7f0fb2762640 1 -- 172.21.15.5:0/3437166435 wait complete. 2024-03-18T15:34:47.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.792+0000 7f0fb2762640 1 Processor -- start 2024-03-18T15:34:47.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.792+0000 7f0fb2762640 1 -- start start 2024-03-18T15:34:47.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.792+0000 7f0fb2762640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0fac06d2d0 0x7f0fac0b7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:47.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.792+0000 7f0fb2762640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fac0ae0f0 0x7f0fac0b6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:47.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.792+0000 7f0fb2762640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0fac0ae9c0 0x7f0fac13ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:47.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.792+0000 7f0fb2762640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0fac14b670 con 0x7f0fac0ae0f0 2024-03-18T15:34:47.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.792+0000 7f0fb2762640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0fac14b4f0 con 0x7f0fac06d2d0 2024-03-18T15:34:47.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.792+0000 7f0fb2762640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f0fac14b7f0 con 0x7f0fac0ae9c0 2024-03-18T15:34:47.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.792+0000 7f0fab7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fac0ae0f0 0x7f0fac0b6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:47.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.792+0000 7f0fab7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fac0ae0f0 0x7f0fac0b6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34292/0 (socket says 172.21.15.5:34292) 2024-03-18T15:34:47.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.792+0000 7f0fab7fe640 1 -- 172.21.15.5:0/2382151533 learned_addr learned my addr 172.21.15.5:0/2382151533 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:47.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.792+0000 7f0fabfff640 1 --2- 172.21.15.5:0/2382151533 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0fac06d2d0 0x7f0fac0b7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:47.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.792+0000 7f0fb0cff640 1 --2- 172.21.15.5:0/2382151533 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0fac0ae9c0 0x7f0fac13ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:47.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.792+0000 7f0fab7fe640 1 -- 172.21.15.5:0/2382151533 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0fac0ae9c0 msgr2=0x7f0fac13ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:47.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.792+0000 7f0fab7fe640 1 --2- 172.21.15.5:0/2382151533 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0fac0ae9c0 0x7f0fac13ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:47.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.792+0000 7f0fab7fe640 1 -- 172.21.15.5:0/2382151533 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0fac06d2d0 msgr2=0x7f0fac0b7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:47.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.792+0000 7f0fab7fe640 1 --2- 172.21.15.5:0/2382151533 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0fac06d2d0 0x7f0fac0b7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:47.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.792+0000 7f0fab7fe640 1 -- 172.21.15.5:0/2382151533 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fac0b8100 con 0x7f0fac0ae0f0 2024-03-18T15:34:47.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.792+0000 7f0fab7fe640 1 --2- 172.21.15.5:0/2382151533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fac0ae0f0 0x7f0fac0b6bd0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c00e840 tx=0x7f0f9c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:47.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.792+0000 7f0fa8ff9640 1 -- 172.21.15.5:0/2382151533 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c017070 con 0x7f0fac0ae0f0 2024-03-18T15:34:47.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.792+0000 7f0fa8ff9640 1 -- 172.21.15.5:0/2382151533 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f9c00c670 con 0x7f0fac0ae0f0 2024-03-18T15:34:47.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.792+0000 7f0fa8ff9640 1 -- 172.21.15.5:0/2382151533 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c00f040 con 0x7f0fac0ae0f0 2024-03-18T15:34:47.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.792+0000 7f0fb2762640 1 -- 172.21.15.5:0/2382151533 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fac0b83d0 con 0x7f0fac0ae0f0 2024-03-18T15:34:47.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.792+0000 7f0fb2762640 1 -- 172.21.15.5:0/2382151533 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fac137120 con 0x7f0fac0ae0f0 2024-03-18T15:34:47.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.792+0000 7f0fb2762640 1 -- 172.21.15.5:0/2382151533 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fac04f550 con 0x7f0fac0ae0f0 2024-03-18T15:34:47.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.792+0000 7f0fa8ff9640 1 -- 172.21.15.5:0/2382151533 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f0f9c01a070 con 0x7f0fac0ae0f0 2024-03-18T15:34:47.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.796+0000 7f0fa8ff9640 1 --2- 172.21.15.5:0/2382151533 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0f98067660 0x7f0f98069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:47.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.796+0000 7f0fa8ff9640 1 -- 172.21.15.5:0/2382151533 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7f0f9c0850b0 con 0x7f0fac0ae0f0 2024-03-18T15:34:47.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.796+0000 7f0fabfff640 1 --2- 172.21.15.5:0/2382151533 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0f98067660 0x7f0f98069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:47.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.796+0000 7f0fabfff640 1 --2- 172.21.15.5:0/2382151533 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0f98067660 0x7f0f98069b20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0fa000ba30 tx=0x7f0fa0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:47.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.800+0000 7f0fa8ff9640 1 -- 172.21.15.5:0/2382151533 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f0f9c056020 con 0x7f0fac0ae0f0 2024-03-18T15:34:47.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.952+0000 7f0fb2762640 1 -- 172.21.15.5:0/2382151533 --> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f0fac072e80 con 0x7f0f98067660 2024-03-18T15:34:47.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.952+0000 7f0fa8ff9640 1 -- 172.21.15.5:0/2382151533 <== mgr.4110 v2:172.21.15.5:6832/3362034860 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f0fac072e80 con 0x7f0f98067660 2024-03-18T15:34:47.957 INFO:teuthology.orchestra.run.smithi005.stderr:instructed osd(s) 7 to deep-scrub 2024-03-18T15:34:47.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.956+0000 7f0fb2762640 1 -- 172.21.15.5:0/2382151533 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0f98067660 msgr2=0x7f0f98069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:47.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.956+0000 7f0fb2762640 1 --2- 172.21.15.5:0/2382151533 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0f98067660 0x7f0f98069b20 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0fa000ba30 tx=0x7f0fa0008040 comp rx=0 tx=0).stop 2024-03-18T15:34:47.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.956+0000 7f0fb2762640 1 -- 172.21.15.5:0/2382151533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fac0ae0f0 msgr2=0x7f0fac0b6bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:47.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.956+0000 7f0fb2762640 1 --2- 172.21.15.5:0/2382151533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fac0ae0f0 0x7f0fac0b6bd0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c00e840 tx=0x7f0f9c00ed00 comp rx=0 tx=0).stop 2024-03-18T15:34:47.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.956+0000 7f0fb2762640 1 -- 172.21.15.5:0/2382151533 shutdown_connections 2024-03-18T15:34:47.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.956+0000 7f0fb2762640 1 --2- 172.21.15.5:0/2382151533 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f0f98067660 0x7f0f98069b20 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:47.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.956+0000 7f0fb2762640 1 --2- 172.21.15.5:0/2382151533 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f0fac0ae9c0 0x7f0fac13ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:47.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.956+0000 7f0fb2762640 1 --2- 172.21.15.5:0/2382151533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0fac0ae0f0 0x7f0fac0b6bd0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:47.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.956+0000 7f0fb2762640 1 --2- 172.21.15.5:0/2382151533 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0fac06d2d0 0x7f0fac0b7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:47.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.956+0000 7f0fb2762640 1 -- 172.21.15.5:0/2382151533 >> 172.21.15.5:0/2382151533 conn(0x7f0fac0b5670 msgr2=0x7f0fac0b5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:47.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.956+0000 7f0fb2762640 1 -- 172.21.15.5:0/2382151533 shutdown_connections 2024-03-18T15:34:47.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:47.956+0000 7f0fb2762640 1 -- 172.21.15.5:0/2382151533 wait complete. 2024-03-18T15:34:47.974 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-18T15:34:48.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.092+0000 7fd577435640 1 Processor -- start 2024-03-18T15:34:48.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.092+0000 7fd577435640 1 -- start start 2024-03-18T15:34:48.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.092+0000 7fd577435640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd57014f9c0 0x7fd570072470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:48.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.092+0000 7fd577435640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd570070360 0x7fd57014b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:48.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.092+0000 7fd577435640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5700707b0 con 0x7fd570072090 2024-03-18T15:34:48.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.092+0000 7fd577435640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fd57014be10 con 0x7fd57014f9c0 2024-03-18T15:34:48.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.092+0000 7fd577435640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fd570071910 con 0x7fd570070360 2024-03-18T15:34:48.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.092+0000 7fd5759d2640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd570070360 0x7fd57014b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:48.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.092+0000 7fd5759d2640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd570070360 0x7fd57014b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3301/0 says I am v2:172.21.15.5:56078/0 (socket says 172.21.15.5:56078) 2024-03-18T15:34:48.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.092+0000 7fd5749d0640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd57014f9c0 0x7fd570072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:48.098 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.092+0000 7fd5759d2640 1 -- 172.21.15.5:0/1091022343 learned_addr learned my addr 172.21.15.5:0/1091022343 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:48.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.092+0000 7fd5751d1640 1 --1- 172.21.15.5:0/1091022343 >> v1:172.21.15.80:6789/0 conn(0x7fd570072090 0x7fd570148a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.5:35966/0 (socket says 172.21.15.5:35966) 2024-03-18T15:34:48.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.096+0000 7fd5749d0640 1 -- 172.21.15.5:0/1091022343 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd570070360 msgr2=0x7fd57014b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:34:48.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.096+0000 7fd5749d0640 1 --2- 172.21.15.5:0/1091022343 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd570070360 0x7fd57014b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:48.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.096+0000 7fd5749d0640 1 -- 172.21.15.5:0/1091022343 >> v1:172.21.15.80:6789/0 conn(0x7fd570072090 legacy=0x7fd570148a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:48.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.096+0000 7fd5749d0640 1 -- 172.21.15.5:0/1091022343 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd57014eee0 con 0x7fd57014f9c0 2024-03-18T15:34:48.099 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.096+0000 7fd5759d2640 1 --2- 172.21.15.5:0/1091022343 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd570070360 0x7fd57014b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-18T15:34:48.100 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.096+0000 7fd5749d0640 1 --2- 172.21.15.5:0/1091022343 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd57014f9c0 0x7fd570072470 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd56c0097c0 tx=0x7fd56c01c380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17e2a4ce61d9a958 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:48.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.096+0000 7fd567fff640 1 -- 172.21.15.5:0/1091022343 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56c0b0070 con 0x7fd57014f9c0 2024-03-18T15:34:48.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.096+0000 7fd567fff640 1 -- 172.21.15.5:0/1091022343 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd56c0ab600 con 0x7fd57014f9c0 2024-03-18T15:34:48.101 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.096+0000 7fd567fff640 1 -- 172.21.15.5:0/1091022343 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56c0ab9d0 con 0x7fd57014f9c0 2024-03-18T15:34:48.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.100+0000 7fd577435640 1 -- 172.21.15.5:0/1091022343 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd57014f9c0 msgr2=0x7fd570072470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:48.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.100+0000 7fd577435640 1 --2- 172.21.15.5:0/1091022343 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd57014f9c0 0x7fd570072470 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd56c0097c0 tx=0x7fd56c01c380 comp rx=0 tx=0).stop 2024-03-18T15:34:48.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.100+0000 7fd577435640 1 -- 172.21.15.5:0/1091022343 shutdown_connections 2024-03-18T15:34:48.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.100+0000 7fd577435640 1 --2- 172.21.15.5:0/1091022343 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd570070360 0x7fd57014b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:48.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.100+0000 7fd577435640 1 --2- 172.21.15.5:0/1091022343 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd57014f9c0 0x7fd570072470 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:48.102 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.100+0000 7fd577435640 1 -- 172.21.15.5:0/1091022343 >> 172.21.15.5:0/1091022343 conn(0x7fd5700b6680 msgr2=0x7fd5700b6a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:48.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.100+0000 7fd577435640 1 -- 172.21.15.5:0/1091022343 shutdown_connections 2024-03-18T15:34:48.103 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.100+0000 7fd577435640 1 -- 172.21.15.5:0/1091022343 wait complete. 2024-03-18T15:34:48.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.100+0000 7fd577435640 1 Processor -- start 2024-03-18T15:34:48.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.100+0000 7fd577435640 1 -- start start 2024-03-18T15:34:48.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.100+0000 7fd577435640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd570070360 0x7fd57014d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:48.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.100+0000 7fd577435640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd570072090 0x7fd57013c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:48.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.100+0000 7fd577435640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd57014f9c0 0x7fd57014aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:48.105 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.100+0000 7fd577435640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd570158990 con 0x7fd570072090 2024-03-18T15:34:48.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.100+0000 7fd577435640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd570158b10 con 0x7fd570070360 2024-03-18T15:34:48.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.100+0000 7fd577435640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fd570158c90 con 0x7fd57014f9c0 2024-03-18T15:34:48.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.100+0000 7fd5749d0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd570072090 0x7fd57013c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:48.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.100+0000 7fd5749d0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd570072090 0x7fd57013c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34296/0 (socket says 172.21.15.5:34296) 2024-03-18T15:34:48.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.100+0000 7fd5749d0640 1 -- 172.21.15.5:0/3062355812 learned_addr learned my addr 172.21.15.5:0/3062355812 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:34:48.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.100+0000 7fd5751d1640 1 --2- 172.21.15.5:0/3062355812 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd570070360 0x7fd57014d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:48.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.100+0000 7fd5759d2640 1 --2- 172.21.15.5:0/3062355812 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd57014f9c0 0x7fd57014aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:48.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.100+0000 7fd5749d0640 1 -- 172.21.15.5:0/3062355812 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd57014f9c0 msgr2=0x7fd57014aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:48.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.100+0000 7fd5749d0640 1 --2- 172.21.15.5:0/3062355812 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd57014f9c0 0x7fd57014aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:48.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.100+0000 7fd5749d0640 1 -- 172.21.15.5:0/3062355812 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd570070360 msgr2=0x7fd57014d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:48.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.100+0000 7fd5749d0640 1 --2- 172.21.15.5:0/3062355812 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd570070360 0x7fd57014d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:48.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.100+0000 7fd5749d0640 1 -- 172.21.15.5:0/3062355812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd57014d060 con 0x7fd570072090 2024-03-18T15:34:48.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.100+0000 7fd5749d0640 1 --2- 172.21.15.5:0/3062355812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd570072090 0x7fd57013c4a0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd56c0ae040 tx=0x7fd56c019120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:48.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.100+0000 7fd565ffb640 1 -- 172.21.15.5:0/3062355812 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56c0ba070 con 0x7fd570072090 2024-03-18T15:34:48.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.100+0000 7fd565ffb640 1 -- 172.21.15.5:0/3062355812 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd56c0b43f0 con 0x7fd570072090 2024-03-18T15:34:48.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.100+0000 7fd565ffb640 1 -- 172.21.15.5:0/3062355812 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56c0b0040 con 0x7fd570072090 2024-03-18T15:34:48.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.100+0000 7fd577435640 1 -- 172.21.15.5:0/3062355812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd57013d080 con 0x7fd570072090 2024-03-18T15:34:48.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.104+0000 7fd577435640 1 -- 172.21.15.5:0/3062355812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5701e7870 con 0x7fd570072090 2024-03-18T15:34:48.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.104+0000 7fd577435640 1 -- 172.21.15.5:0/3062355812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd57004f550 con 0x7fd570072090 2024-03-18T15:34:48.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.104+0000 7fd565ffb640 1 -- 172.21.15.5:0/3062355812 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fd56c0bd070 con 0x7fd570072090 2024-03-18T15:34:48.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.104+0000 7fd565ffb640 1 --2- 172.21.15.5:0/3062355812 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd554067660 0x7fd554069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:34:48.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.104+0000 7fd565ffb640 1 -- 172.21.15.5:0/3062355812 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7fd56c11e5b0 con 0x7fd570072090 2024-03-18T15:34:48.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.104+0000 7fd5751d1640 1 --2- 172.21.15.5:0/3062355812 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd554067660 0x7fd554069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:34:48.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.104+0000 7fd5751d1640 1 --2- 172.21.15.5:0/3062355812 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd554067660 0x7fd554069b20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd560000f70 tx=0x7fd560007e90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:34:48.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.108+0000 7fd565ffb640 1 -- 172.21.15.5:0/3062355812 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fd56c016090 con 0x7fd570072090 2024-03-18T15:34:48.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.268+0000 7fd577435640 1 -- 172.21.15.5:0/3062355812 --> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd570145f80 con 0x7fd554067660 2024-03-18T15:34:48.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.268+0000 7fd565ffb640 1 -- 172.21.15.5:0/3062355812 <== mgr.4110 v2:172.21.15.5:6832/3362034860 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13478 (secure 0 0 0) 0x7fd570145f80 con 0x7fd554067660 2024-03-18T15:34:48.271 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:34:48.272 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-03-18T15:34:48.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.272+0000 7fd577435640 1 -- 172.21.15.5:0/3062355812 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd554067660 msgr2=0x7fd554069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:48.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.272+0000 7fd577435640 1 --2- 172.21.15.5:0/3062355812 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd554067660 0x7fd554069b20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd560000f70 tx=0x7fd560007e90 comp rx=0 tx=0).stop 2024-03-18T15:34:48.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.272+0000 7fd577435640 1 -- 172.21.15.5:0/3062355812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd570072090 msgr2=0x7fd57013c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:34:48.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.272+0000 7fd577435640 1 --2- 172.21.15.5:0/3062355812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd570072090 0x7fd57013c4a0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd56c0ae040 tx=0x7fd56c019120 comp rx=0 tx=0).stop 2024-03-18T15:34:48.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.272+0000 7fd577435640 1 -- 172.21.15.5:0/3062355812 shutdown_connections 2024-03-18T15:34:48.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.272+0000 7fd577435640 1 --2- 172.21.15.5:0/3062355812 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fd554067660 0x7fd554069b20 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:48.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.272+0000 7fd577435640 1 --2- 172.21.15.5:0/3062355812 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fd57014f9c0 0x7fd57014aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:48.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.272+0000 7fd577435640 1 --2- 172.21.15.5:0/3062355812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd570072090 0x7fd57013c4a0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:48.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.272+0000 7fd577435640 1 --2- 172.21.15.5:0/3062355812 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd570070360 0x7fd57014d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:34:48.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.272+0000 7fd577435640 1 -- 172.21.15.5:0/3062355812 >> 172.21.15.5:0/3062355812 conn(0x7fd5700b6680 msgr2=0x7fd570148850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:34:48.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.272+0000 7fd577435640 1 -- 172.21.15.5:0/3062355812 shutdown_connections 2024-03-18T15:34:48.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:34:48.272+0000 7fd577435640 1 -- 172.21.15.5:0/3062355812 wait complete. 2024-03-18T15:34:48.290 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2024-03-18T15:34:46.903151+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":21,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":253532,"kb_used_data":4072,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754721188,"statfs":{"total":773094113280,"available":772834496512,"internally_reserved":0,"allocated":4169728,"data_stored":2334192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10891,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-54,"num_write_kb":-26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001979"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":194,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-18T15:34:40.225869+0000","last_change":"2024-03-18T15:33:30.766315+0000","last_active":"2024-03-18T15:34:40.225869+0000","last_peered":"2024-03-18T15:34:40.225869+0000","last_clean":"2024-03-18T15:34:40.225869+0000","last_became_active":"2024-03-18T15:33:30.763939+0000","last_became_peered":"2024-03-18T15:33:30.763939+0000","last_unstale":"2024-03-18T15:34:40.225869+0000","last_undegraded":"2024-03-18T15:34:40.225869+0000","last_fullsized":"2024-03-18T15:34:40.225869+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-18T15:33:29.229911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-18T15:33:29.229911+0000","last_clean_scrub_stamp":"2024-03-18T15:33:29.229911+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T21:10:21.120371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509459,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27384,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":372736,"data_stored":144182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509459,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27384,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":372736,"data_stored":144182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542163,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":966656,"data_stored":734550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542164,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27388,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":372736,"data_stored":144182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542162,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35708,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336132,"statfs":{"total":96636764160,"available":96600199168,"internally_reserved":0,"allocated":372736,"data_stored":144182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542162,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35704,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336136,"statfs":{"total":96636764160,"available":96600203264,"internally_reserved":0,"allocated":372736,"data_stored":144182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542162,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35708,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336132,"statfs":{"total":96636764160,"available":96600199168,"internally_reserved":0,"allocated":372736,"data_stored":144182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542161,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36288,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335552,"statfs":{"total":96636764160,"available":96599605248,"internally_reserved":0,"allocated":966656,"data_stored":734550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-18T15:34:48.293 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-03-18T15:33:29.229911+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=18, tm_hour=15, tm_min=33, tm_sec=29, tm_wday=0, tm_yday=78, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=18, tm_hour=15, tm_min=34, tm_sec=43, tm_wday=0, tm_yday=78, tm_isdst=0) 2024-03-18T15:34:48.293 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-03-18T15:35:08.294 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-18T15:35:08.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.423+0000 7fb989c5e640 1 Processor -- start 2024-03-18T15:35:08.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.423+0000 7fb989c5e640 1 -- start start 2024-03-18T15:35:08.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.427+0000 7fb989c5e640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb98406d2d0 0x7fb98414bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:35:08.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.427+0000 7fb989c5e640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb9840b4780 0x7fb98414c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:35:08.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.427+0000 7fb989c5e640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb9840b2690 0x7fb9840b2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:35:08.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.427+0000 7fb989c5e640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7fb9840aeb70 con 0x7fb9840b4780 2024-03-18T15:35:08.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.427+0000 7fb989c5e640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fb98406d6b0 con 0x7fb9840b2690 2024-03-18T15:35:08.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.427+0000 7fb989c5e640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fb9840b3030 con 0x7fb98406d2d0 2024-03-18T15:35:08.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.427+0000 7fb982ffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb9840b4780 0x7fb98414c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:35:08.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.427+0000 7fb982ffd640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7fb9840b4780 0x7fb98414c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48436/0 (socket says 172.21.15.5:48436) 2024-03-18T15:35:08.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.427+0000 7fb982ffd640 1 -- 172.21.15.5:0/2771300673 learned_addr learned my addr 172.21.15.5:0/2771300673 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:35:08.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.427+0000 7fb9837fe640 1 --2- 172.21.15.5:0/2771300673 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb98406d2d0 0x7fb98414bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:35:08.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.427+0000 7fb983fff640 1 --2- 172.21.15.5:0/2771300673 >> v2:172.21.15.80:3300/0 conn(0x7fb9840b2690 0x7fb9840b2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:35:08.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.427+0000 7fb982ffd640 1 -- 172.21.15.5:0/2771300673 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb98406d2d0 msgr2=0x7fb98414bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:35:08.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.427+0000 7fb982ffd640 1 --2- 172.21.15.5:0/2771300673 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb98406d2d0 0x7fb98414bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:08.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.427+0000 7fb982ffd640 1 -- 172.21.15.5:0/2771300673 >> v2:172.21.15.80:3300/0 conn(0x7fb9840b2690 msgr2=0x7fb9840b2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:35:08.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.427+0000 7fb982ffd640 1 --2- 172.21.15.5:0/2771300673 >> v2:172.21.15.80:3300/0 conn(0x7fb9840b2690 0x7fb9840b2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:08.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.427+0000 7fb982ffd640 1 -- 172.21.15.5:0/2771300673 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9840ae520 con 0x7fb9840b4780 2024-03-18T15:35:08.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.427+0000 7fb982ffd640 1 --2- 172.21.15.5:0/2771300673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9840b4780 0x7fb98414c240 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb9840b4710 tx=0x7fb97801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9784e9b650065938 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:35:08.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.427+0000 7fb9827fc640 1 -- 172.21.15.5:0/2771300673 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9780ab650 con 0x7fb9840b4780 2024-03-18T15:35:08.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.427+0000 7fb9827fc640 1 -- 172.21.15.5:0/2771300673 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9780b0070 con 0x7fb9840b4780 2024-03-18T15:35:08.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.427+0000 7fb9827fc640 1 -- 172.21.15.5:0/2771300673 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9780aba10 con 0x7fb9840b4780 2024-03-18T15:35:08.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.431+0000 7fb989c5e640 1 -- 172.21.15.5:0/2771300673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9840b4780 msgr2=0x7fb98414c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:35:08.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.431+0000 7fb989c5e640 1 --2- 172.21.15.5:0/2771300673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9840b4780 0x7fb98414c240 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb9840b4710 tx=0x7fb97801c490 comp rx=0 tx=0).stop 2024-03-18T15:35:08.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.431+0000 7fb989c5e640 1 -- 172.21.15.5:0/2771300673 shutdown_connections 2024-03-18T15:35:08.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.431+0000 7fb989c5e640 1 --2- 172.21.15.5:0/2771300673 >> v2:172.21.15.80:3300/0 conn(0x7fb9840b2690 0x7fb9840b2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:08.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.431+0000 7fb989c5e640 1 --2- 172.21.15.5:0/2771300673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9840b4780 0x7fb98414c240 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:08.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.431+0000 7fb989c5e640 1 --2- 172.21.15.5:0/2771300673 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb98406d2d0 0x7fb98414bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:08.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.431+0000 7fb989c5e640 1 -- 172.21.15.5:0/2771300673 >> 172.21.15.5:0/2771300673 conn(0x7fb9840b5670 msgr2=0x7fb9840b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:35:08.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.431+0000 7fb989c5e640 1 -- 172.21.15.5:0/2771300673 shutdown_connections 2024-03-18T15:35:08.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.431+0000 7fb989c5e640 1 -- 172.21.15.5:0/2771300673 wait complete. 2024-03-18T15:35:08.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.431+0000 7fb989c5e640 1 Processor -- start 2024-03-18T15:35:08.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.431+0000 7fb989c5e640 1 -- start start 2024-03-18T15:35:08.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.431+0000 7fb989c5e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb98406d2d0 0x7fb98414abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:35:08.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.431+0000 7fb989c5e640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb9840b2690 0x7fb9841475c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:35:08.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.431+0000 7fb989c5e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9840b4780 0x7fb984147b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:35:08.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.431+0000 7fb989c5e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb9840ae300 con 0x7fb98406d2d0 2024-03-18T15:35:08.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.431+0000 7fb989c5e640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb9840ae180 con 0x7fb9840b4780 2024-03-18T15:35:08.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.431+0000 7fb989c5e640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7fb98414f730 con 0x7fb9840b2690 2024-03-18T15:35:08.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.431+0000 7fb9837fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb98406d2d0 0x7fb98414abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:35:08.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.431+0000 7fb9837fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb98406d2d0 0x7fb98414abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48448/0 (socket says 172.21.15.5:48448) 2024-03-18T15:35:08.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.431+0000 7fb9837fe640 1 -- 172.21.15.5:0/3553358426 learned_addr learned my addr 172.21.15.5:0/3553358426 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:35:08.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.431+0000 7fb983fff640 1 --2- 172.21.15.5:0/3553358426 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9840b4780 0x7fb984147b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:35:08.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.431+0000 7fb982ffd640 1 --2- 172.21.15.5:0/3553358426 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb9840b2690 0x7fb9841475c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:35:08.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.431+0000 7fb9837fe640 1 -- 172.21.15.5:0/3553358426 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb9840b2690 msgr2=0x7fb9841475c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:35:08.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.431+0000 7fb9837fe640 1 --2- 172.21.15.5:0/3553358426 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb9840b2690 0x7fb9841475c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:08.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.431+0000 7fb9837fe640 1 -- 172.21.15.5:0/3553358426 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9840b4780 msgr2=0x7fb984147b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:35:08.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.431+0000 7fb9837fe640 1 --2- 172.21.15.5:0/3553358426 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9840b4780 0x7fb984147b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:08.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.431+0000 7fb9837fe640 1 -- 172.21.15.5:0/3553358426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb984148040 con 0x7fb98406d2d0 2024-03-18T15:35:08.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.431+0000 7fb983fff640 1 --2- 172.21.15.5:0/3553358426 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9840b4780 0x7fb984147b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-18T15:35:08.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.431+0000 7fb9837fe640 1 --2- 172.21.15.5:0/3553358426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb98406d2d0 0x7fb98414abd0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb97c007e60 tx=0x7fb97c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:35:08.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.431+0000 7fb96bfff640 1 -- 172.21.15.5:0/3553358426 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97c014620 con 0x7fb98406d2d0 2024-03-18T15:35:08.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.431+0000 7fb96bfff640 1 -- 172.21.15.5:0/3553358426 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb97c019070 con 0x7fb98406d2d0 2024-03-18T15:35:08.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.431+0000 7fb989c5e640 1 -- 172.21.15.5:0/3553358426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb984148db0 con 0x7fb98406d2d0 2024-03-18T15:35:08.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.431+0000 7fb96bfff640 1 -- 172.21.15.5:0/3553358426 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97c014a00 con 0x7fb98406d2d0 2024-03-18T15:35:08.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.431+0000 7fb989c5e640 1 -- 172.21.15.5:0/3553358426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb984145460 con 0x7fb98406d2d0 2024-03-18T15:35:08.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.435+0000 7fb989c5e640 1 -- 172.21.15.5:0/3553358426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb98404f550 con 0x7fb98406d2d0 2024-03-18T15:35:08.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.435+0000 7fb96bfff640 1 -- 172.21.15.5:0/3553358426 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7fb97c014ba0 con 0x7fb98406d2d0 2024-03-18T15:35:08.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.435+0000 7fb96bfff640 1 --2- 172.21.15.5:0/3553358426 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fb970067690 0x7fb970069b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:35:08.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.435+0000 7fb96bfff640 1 -- 172.21.15.5:0/3553358426 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7fb97c085d90 con 0x7fb98406d2d0 2024-03-18T15:35:08.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.435+0000 7fb982ffd640 1 --2- 172.21.15.5:0/3553358426 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fb970067690 0x7fb970069b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:35:08.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.435+0000 7fb982ffd640 1 --2- 172.21.15.5:0/3553358426 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fb970067690 0x7fb970069b50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb9780af000 tx=0x7fb9780a7000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:35:08.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.439+0000 7fb96bfff640 1 -- 172.21.15.5:0/3553358426 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7fb97c056da0 con 0x7fb98406d2d0 2024-03-18T15:35:08.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.595+0000 7fb989c5e640 1 -- 172.21.15.5:0/3553358426 --> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb9840af390 con 0x7fb970067690 2024-03-18T15:35:08.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.595+0000 7fb96bfff640 1 -- 172.21.15.5:0/3553358426 <== mgr.4110 v2:172.21.15.5:6832/3362034860 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13354 (secure 0 0 0) 0x7fb9840af390 con 0x7fb970067690 2024-03-18T15:35:08.601 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:35:08.602 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-03-18T15:35:08.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.599+0000 7fb989c5e640 1 -- 172.21.15.5:0/3553358426 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fb970067690 msgr2=0x7fb970069b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:35:08.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.599+0000 7fb989c5e640 1 --2- 172.21.15.5:0/3553358426 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fb970067690 0x7fb970069b50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb9780af000 tx=0x7fb9780a7000 comp rx=0 tx=0).stop 2024-03-18T15:35:08.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.599+0000 7fb989c5e640 1 -- 172.21.15.5:0/3553358426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb98406d2d0 msgr2=0x7fb98414abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:35:08.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.599+0000 7fb989c5e640 1 --2- 172.21.15.5:0/3553358426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb98406d2d0 0x7fb98414abd0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb97c007e60 tx=0x7fb97c00e4b0 comp rx=0 tx=0).stop 2024-03-18T15:35:08.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.599+0000 7fb989c5e640 1 -- 172.21.15.5:0/3553358426 shutdown_connections 2024-03-18T15:35:08.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.599+0000 7fb989c5e640 1 --2- 172.21.15.5:0/3553358426 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7fb970067690 0x7fb970069b50 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:08.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.599+0000 7fb989c5e640 1 --2- 172.21.15.5:0/3553358426 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9840b4780 0x7fb984147b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:08.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.599+0000 7fb989c5e640 1 --2- 172.21.15.5:0/3553358426 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7fb9840b2690 0x7fb9841475c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:08.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.599+0000 7fb989c5e640 1 --2- 172.21.15.5:0/3553358426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb98406d2d0 0x7fb98414abd0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:08.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.599+0000 7fb989c5e640 1 -- 172.21.15.5:0/3553358426 >> 172.21.15.5:0/3553358426 conn(0x7fb9840b5670 msgr2=0x7fb984146400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:35:08.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.599+0000 7fb989c5e640 1 -- 172.21.15.5:0/3553358426 shutdown_connections 2024-03-18T15:35:08.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.599+0000 7fb989c5e640 1 -- 172.21.15.5:0/3553358426 wait complete. 2024-03-18T15:35:08.619 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2024-03-18T15:35:06.908361+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":253516,"kb_used_data":4072,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754721204,"statfs":{"total":773094113280,"available":772834512896,"internally_reserved":0,"allocated":4169728,"data_stored":2334192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003130"},"pg_stats":[{"pgid":"1.0","version":"33'119","reported_seq":205,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-18T15:34:48.689852+0000","last_change":"2024-03-18T15:34:48.689953+0000","last_active":"2024-03-18T15:34:48.689852+0000","last_peered":"2024-03-18T15:34:48.689852+0000","last_clean":"2024-03-18T15:34:48.689852+0000","last_became_active":"2024-03-18T15:33:30.763939+0000","last_became_peered":"2024-03-18T15:33:30.763939+0000","last_unstale":"2024-03-18T15:34:48.689852+0000","last_undegraded":"2024-03-18T15:34:48.689852+0000","last_fullsized":"2024-03-18T15:34:48.689852+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'119","last_scrub_stamp":"2024-03-18T15:34:48.689653+0000","last_deep_scrub":"33'119","last_deep_scrub_stamp":"2024-03-18T15:34:48.689653+0000","last_clean_scrub_stamp":"2024-03-18T15:34:48.689653+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-19T19:55:56.153120+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509463,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27384,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":372736,"data_stored":144182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509463,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27384,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":372736,"data_stored":144182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542167,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":966656,"data_stored":734550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542168,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27388,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":372736,"data_stored":144182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542166,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35708,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336132,"statfs":{"total":96636764160,"available":96600199168,"internally_reserved":0,"allocated":372736,"data_stored":144182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542166,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35704,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336136,"statfs":{"total":96636764160,"available":96600203264,"internally_reserved":0,"allocated":372736,"data_stored":144182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542166,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":35708,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336132,"statfs":{"total":96636764160,"available":96600199168,"internally_reserved":0,"allocated":372736,"data_stored":144182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542165,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36280,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335560,"statfs":{"total":96636764160,"available":96599613440,"internally_reserved":0,"allocated":966656,"data_stored":734550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-18T15:35:08.620 DEBUG:teuthology.orchestra.run.smithi005:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-03-18T15:35:08.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.739+0000 7f70494d8640 1 Processor -- start 2024-03-18T15:35:08.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.739+0000 7f70494d8640 1 -- start start 2024-03-18T15:35:08.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.739+0000 7f70494d8640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f70440b1bc0 0x7f704406d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:35:08.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.739+0000 7f70494d8640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7044141b20 0x7f70440b6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:35:08.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.739+0000 7f70494d8640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70440b7d80 con 0x7f70440b79a0 2024-03-18T15:35:08.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.739+0000 7f70494d8640 1 -- --> v2:172.21.15.5:3300/0 -- mon_getmap magic: 0 -- 0x7f70440b6250 con 0x7f70440b1bc0 2024-03-18T15:35:08.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.739+0000 7f70494d8640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f70440b6530 con 0x7f7044141b20 2024-03-18T15:35:08.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.739+0000 7f70427fc640 1 --2- >> v2:172.21.15.5:3300/0 conn(0x7f70440b1bc0 0x7f704406d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:35:08.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.739+0000 7f7042ffd640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f70440b79a0 0x7f7044141410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38630/0 (socket says 172.21.15.5:38630) 2024-03-18T15:35:08.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.739+0000 7f7042ffd640 1 -- 172.21.15.5:0/2738599316 learned_addr learned my addr 172.21.15.5:0/2738599316 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:35:08.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.739+0000 7f70437fe640 1 --2- 172.21.15.5:0/2738599316 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7044141b20 0x7f70440b6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:35:08.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.739+0000 7f70427fc640 1 -- 172.21.15.5:0/2738599316 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7044141b20 msgr2=0x7f70440b6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-18T15:35:08.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.739+0000 7f70427fc640 1 --2- 172.21.15.5:0/2738599316 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7044141b20 0x7f70440b6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:08.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.739+0000 7f70427fc640 1 -- 172.21.15.5:0/2738599316 >> v1:172.21.15.5:6789/0 conn(0x7f70440b79a0 legacy=0x7f7044141410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:35:08.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.739+0000 7f7041ffb640 1 -- 172.21.15.5:0/2738599316 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2221976583 0 0) 0x7f70440b7d80 con 0x7f70440b79a0 2024-03-18T15:35:08.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.739+0000 7f70427fc640 1 -- 172.21.15.5:0/2738599316 --> v2:172.21.15.5:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70440b56b0 con 0x7f70440b1bc0 2024-03-18T15:35:08.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.743+0000 7f70427fc640 1 --2- 172.21.15.5:0/2738599316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70440b1bc0 0x7f704406d310 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f70380097c0 tx=0x7f703801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31e77e5b62940f70 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:35:08.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.743+0000 7f7041ffb640 1 -- 172.21.15.5:0/2738599316 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70380b1070 con 0x7f70440b1bc0 2024-03-18T15:35:08.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.743+0000 7f7041ffb640 1 -- 172.21.15.5:0/2738599316 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70380ac410 con 0x7f70440b1bc0 2024-03-18T15:35:08.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.743+0000 7f7041ffb640 1 -- 172.21.15.5:0/2738599316 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70380ac7e0 con 0x7f70440b1bc0 2024-03-18T15:35:08.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.743+0000 7f70494d8640 1 -- 172.21.15.5:0/2738599316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70440b1bc0 msgr2=0x7f704406d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:35:08.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.743+0000 7f70494d8640 1 --2- 172.21.15.5:0/2738599316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70440b1bc0 0x7f704406d310 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f70380097c0 tx=0x7f703801c4a0 comp rx=0 tx=0).stop 2024-03-18T15:35:08.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.743+0000 7f70494d8640 1 -- 172.21.15.5:0/2738599316 shutdown_connections 2024-03-18T15:35:08.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.743+0000 7f70494d8640 1 --2- 172.21.15.5:0/2738599316 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f7044141b20 0x7f70440b6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:08.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.743+0000 7f70494d8640 1 --2- 172.21.15.5:0/2738599316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f70440b1bc0 0x7f704406d310 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:08.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.743+0000 7f70494d8640 1 -- 172.21.15.5:0/2738599316 >> 172.21.15.5:0/2738599316 conn(0x7f7044066fc0 msgr2=0x7f70440673d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:35:08.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.743+0000 7f70494d8640 1 -- 172.21.15.5:0/2738599316 shutdown_connections 2024-03-18T15:35:08.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.743+0000 7f70494d8640 1 -- 172.21.15.5:0/2738599316 wait complete. 2024-03-18T15:35:08.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.743+0000 7f70494d8640 1 Processor -- start 2024-03-18T15:35:08.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.743+0000 7f70494d8640 1 -- start start 2024-03-18T15:35:08.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.743+0000 7f70494d8640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f70440b79a0 0x7f704414a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:35:08.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.743+0000 7f70494d8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7044141b20 0x7f70441dfb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:35:08.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.743+0000 7f70494d8640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f70441e00d0 0x7f7044135e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:35:08.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.743+0000 7f70494d8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f70441548b0 con 0x7f7044141b20 2024-03-18T15:35:08.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.743+0000 7f70494d8640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7044072cf0 con 0x7f70440b79a0 2024-03-18T15:35:08.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.743+0000 7f70494d8640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f7044154a30 con 0x7f70441e00d0 2024-03-18T15:35:08.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.743+0000 7f70427fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7044141b20 0x7f70441dfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:35:08.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.743+0000 7f70427fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7044141b20 0x7f70441dfb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48468/0 (socket says 172.21.15.5:48468) 2024-03-18T15:35:08.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.743+0000 7f70427fc640 1 -- 172.21.15.5:0/2138132821 learned_addr learned my addr 172.21.15.5:0/2138132821 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:35:08.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.743+0000 7f7042ffd640 1 --2- 172.21.15.5:0/2138132821 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f70440b79a0 0x7f704414a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:35:08.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.743+0000 7f70437fe640 1 --2- 172.21.15.5:0/2138132821 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f70441e00d0 0x7f7044135e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:35:08.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.743+0000 7f70427fc640 1 -- 172.21.15.5:0/2138132821 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f70441e00d0 msgr2=0x7f7044135e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:35:08.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.743+0000 7f70427fc640 1 --2- 172.21.15.5:0/2138132821 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f70441e00d0 0x7f7044135e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:08.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.743+0000 7f70427fc640 1 -- 172.21.15.5:0/2138132821 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f70440b79a0 msgr2=0x7f704414a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:35:08.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.743+0000 7f70427fc640 1 --2- 172.21.15.5:0/2138132821 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f70440b79a0 0x7f704414a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:08.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.743+0000 7f70427fc640 1 -- 172.21.15.5:0/2138132821 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70441e0550 con 0x7f7044141b20 2024-03-18T15:35:08.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.743+0000 7f70437fe640 1 --2- 172.21.15.5:0/2138132821 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f70441e00d0 0x7f7044135e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:35:08.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.743+0000 7f70427fc640 1 --2- 172.21.15.5:0/2138132821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7044141b20 0x7f70441dfb90 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f703801a0d0 tx=0x7f7038019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:35:08.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.743+0000 7f702b7fe640 1 -- 172.21.15.5:0/2138132821 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70380b9020 con 0x7f7044141b20 2024-03-18T15:35:08.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.743+0000 7f702b7fe640 1 -- 172.21.15.5:0/2138132821 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70380b5410 con 0x7f7044141b20 2024-03-18T15:35:08.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.743+0000 7f70494d8640 1 -- 172.21.15.5:0/2138132821 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70441449c0 con 0x7f7044141b20 2024-03-18T15:35:08.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.743+0000 7f702b7fe640 1 -- 172.21.15.5:0/2138132821 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70380b1040 con 0x7f7044141b20 2024-03-18T15:35:08.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.743+0000 7f70494d8640 1 -- 172.21.15.5:0/2138132821 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7044143c40 con 0x7f7044141b20 2024-03-18T15:35:08.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.747+0000 7f70494d8640 1 -- 172.21.15.5:0/2138132821 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f704404f550 con 0x7f7044141b20 2024-03-18T15:35:08.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.747+0000 7f702b7fe640 1 -- 172.21.15.5:0/2138132821 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f70380bc020 con 0x7f7044141b20 2024-03-18T15:35:08.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.747+0000 7f702b7fe640 1 --2- 172.21.15.5:0/2138132821 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f7030067660 0x7f7030069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:35:08.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.747+0000 7f7042ffd640 1 --2- 172.21.15.5:0/2138132821 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f7030067660 0x7f7030069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:35:08.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.747+0000 7f702b7fe640 1 -- 172.21.15.5:0/2138132821 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7f703811f220 con 0x7f7044141b20 2024-03-18T15:35:08.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.747+0000 7f7042ffd640 1 --2- 172.21.15.5:0/2138132821 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f7030067660 0x7f7030069b20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7034000f70 tx=0x7f7034007b80 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:35:08.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.751+0000 7f702b7fe640 1 -- 172.21.15.5:0/2138132821 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f70380c3030 con 0x7f7044141b20 2024-03-18T15:35:08.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.911+0000 7f70494d8640 1 -- 172.21.15.5:0/2138132821 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f70440723d0 con 0x7f7044141b20 2024-03-18T15:35:08.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.919+0000 7f702b7fe640 1 -- 172.21.15.5:0/2138132821 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) ==== 123+0+0 (secure 0 0 0) 0x7f70441e0550 con 0x7f7044141b20 2024-03-18T15:35:08.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.919+0000 7f702b7fe640 1 -- 172.21.15.5:0/2138132821 <== mon.0 v2:172.21.15.5:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f70380f0190 con 0x7f7044141b20 2024-03-18T15:35:08.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.919+0000 7f70494d8640 1 -- 172.21.15.5:0/2138132821 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f7030067660 msgr2=0x7f7030069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:35:08.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.919+0000 7f70494d8640 1 --2- 172.21.15.5:0/2138132821 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f7030067660 0x7f7030069b20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7034000f70 tx=0x7f7034007b80 comp rx=0 tx=0).stop 2024-03-18T15:35:08.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.919+0000 7f70494d8640 1 -- 172.21.15.5:0/2138132821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7044141b20 msgr2=0x7f70441dfb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:35:08.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.919+0000 7f70494d8640 1 --2- 172.21.15.5:0/2138132821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7044141b20 0x7f70441dfb90 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f703801a0d0 tx=0x7f7038019240 comp rx=0 tx=0).stop 2024-03-18T15:35:08.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.919+0000 7f70494d8640 1 -- 172.21.15.5:0/2138132821 shutdown_connections 2024-03-18T15:35:08.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.919+0000 7f70494d8640 1 --2- 172.21.15.5:0/2138132821 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f7030067660 0x7f7030069b20 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:08.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.919+0000 7f70494d8640 1 --2- 172.21.15.5:0/2138132821 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f70441e00d0 0x7f7044135e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:08.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.919+0000 7f70494d8640 1 --2- 172.21.15.5:0/2138132821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7044141b20 0x7f70441dfb90 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:08.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.919+0000 7f70494d8640 1 --2- 172.21.15.5:0/2138132821 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f70440b79a0 0x7f704414a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:08.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.919+0000 7f70494d8640 1 -- 172.21.15.5:0/2138132821 >> 172.21.15.5:0/2138132821 conn(0x7f7044066fc0 msgr2=0x7f70440b8bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:35:08.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.919+0000 7f70494d8640 1 -- 172.21.15.5:0/2138132821 shutdown_connections 2024-03-18T15:35:08.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:08.919+0000 7f70494d8640 1 -- 172.21.15.5:0/2138132821 wait complete. 2024-03-18T15:35:08.943 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-18T15:35:08.943 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-18T15:35:09.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.059+0000 7f3539c5a640 1 Processor -- start 2024-03-18T15:35:09.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.063+0000 7f3539c5a640 1 -- start start 2024-03-18T15:35:09.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.063+0000 7f3539c5a640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f353413f260 0x7f353413f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:35:09.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.063+0000 7f3539c5a640 1 -- --> v1:172.21.15.5:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35340b2ab0 con 0x7f353414dc70 2024-03-18T15:35:09.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.063+0000 7f3539c5a640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35340b0ea0 con 0x7f353406d2d0 2024-03-18T15:35:09.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.063+0000 7f3539c5a640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f35340728c0 con 0x7f353413f260 2024-03-18T15:35:09.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.063+0000 7f3533fff640 1 --1- >> v1:172.21.15.5:6789/0 conn(0x7f353414dc70 0x7f3534150060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.5:6789/0 says I am v1:172.21.15.5:38644/0 (socket says 172.21.15.5:38644) 2024-03-18T15:35:09.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.063+0000 7f3533fff640 1 -- 172.21.15.5:0/2776663121 learned_addr learned my addr 172.21.15.5:0/2776663121 (peer_addr_for_me v1:172.21.15.5:0/0) 2024-03-18T15:35:09.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.063+0000 7f3532ffd640 1 --2- 172.21.15.5:0/2776663121 >> v2:172.21.15.80:3300/0 conn(0x7f353413f260 0x7f353413f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:35:09.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.063+0000 7f35327fc640 1 -- 172.21.15.5:0/2776663121 <== mon.0 v1:172.21.15.5:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1412888013 0 0) 0x7f35340b2ab0 con 0x7f353414dc70 2024-03-18T15:35:09.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.063+0000 7f35327fc640 1 -- 172.21.15.5:0/2776663121 --> v1:172.21.15.5:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3514003530 con 0x7f353414dc70 2024-03-18T15:35:09.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.063+0000 7f3532ffd640 1 -- 172.21.15.5:0/2776663121 >> v1:172.21.15.80:6789/0 conn(0x7f353406d2d0 legacy=0x7f35340b23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:35:09.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.063+0000 7f3532ffd640 1 -- 172.21.15.5:0/2776663121 >> v1:172.21.15.5:6789/0 conn(0x7f353414dc70 legacy=0x7f3534150060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:35:09.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.063+0000 7f3532ffd640 1 -- 172.21.15.5:0/2776663121 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f353406d6b0 con 0x7f353413f260 2024-03-18T15:35:09.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.067+0000 7f3532ffd640 1 --2- 172.21.15.5:0/2776663121 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f353413f260 0x7f353413f6a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f351c009d90 tx=0x7f351c01c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3330d00cc6cbed4 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:35:09.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.067+0000 7f35327fc640 1 -- 172.21.15.5:0/2776663121 <== mon.1 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351c0b2070 con 0x7f353413f260 2024-03-18T15:35:09.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.067+0000 7f35327fc640 1 -- 172.21.15.5:0/2776663121 <== mon.1 v2:172.21.15.80:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f351c0a5890 con 0x7f353413f260 2024-03-18T15:35:09.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.067+0000 7f35327fc640 1 -- 172.21.15.5:0/2776663121 <== mon.1 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351c0a5c40 con 0x7f353413f260 2024-03-18T15:35:09.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.067+0000 7f3539c5a640 1 -- 172.21.15.5:0/2776663121 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f353413f260 msgr2=0x7f353413f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:35:09.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.067+0000 7f3539c5a640 1 --2- 172.21.15.5:0/2776663121 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f353413f260 0x7f353413f6a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f351c009d90 tx=0x7f351c01c8e0 comp rx=0 tx=0).stop 2024-03-18T15:35:09.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.067+0000 7f3539c5a640 1 -- 172.21.15.5:0/2776663121 shutdown_connections 2024-03-18T15:35:09.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.067+0000 7f3539c5a640 1 --2- 172.21.15.5:0/2776663121 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f353413f260 0x7f353413f6a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:09.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.067+0000 7f3539c5a640 1 -- 172.21.15.5:0/2776663121 >> 172.21.15.5:0/2776663121 conn(0x7f35340b5670 msgr2=0x7f35340b5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:35:09.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.067+0000 7f3539c5a640 1 -- 172.21.15.5:0/2776663121 shutdown_connections 2024-03-18T15:35:09.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.067+0000 7f3539c5a640 1 -- 172.21.15.5:0/2776663121 wait complete. 2024-03-18T15:35:09.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.067+0000 7f3539c5a640 1 Processor -- start 2024-03-18T15:35:09.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.067+0000 7f3539c5a640 1 -- start start 2024-03-18T15:35:09.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.067+0000 7f3539c5a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f353406d2d0 0x7f35341304e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:35:09.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.067+0000 7f3539c5a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f353413f260 0x7f35340adcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:35:09.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.067+0000 7f3539c5a640 1 --2- >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f353414dc70 0x7f35340ae200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:35:09.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.067+0000 7f3539c5a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f35340b0960 con 0x7f353413f260 2024-03-18T15:35:09.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.067+0000 7f3539c5a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f35340b07e0 con 0x7f353406d2d0 2024-03-18T15:35:09.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.067+0000 7f3539c5a640 1 -- --> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] -- mon_getmap magic: 0 -- 0x7f35340b5140 con 0x7f353414dc70 2024-03-18T15:35:09.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.067+0000 7f3532ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f353413f260 0x7f35340adcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:35:09.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.067+0000 7f3532ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f353413f260 0x7f35340adcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48482/0 (socket says 172.21.15.5:48482) 2024-03-18T15:35:09.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.067+0000 7f3532ffd640 1 -- 172.21.15.5:0/727958608 learned_addr learned my addr 172.21.15.5:0/727958608 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-03-18T15:35:09.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.067+0000 7f35337fe640 1 --2- 172.21.15.5:0/727958608 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f353406d2d0 0x7f35341304e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:35:09.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.067+0000 7f3533fff640 1 --2- 172.21.15.5:0/727958608 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f353414dc70 0x7f35340ae200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:35:09.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.067+0000 7f3532ffd640 1 -- 172.21.15.5:0/727958608 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f353414dc70 msgr2=0x7f35340ae200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:35:09.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.067+0000 7f3532ffd640 1 --2- 172.21.15.5:0/727958608 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f353414dc70 0x7f35340ae200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:09.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.067+0000 7f3532ffd640 1 -- 172.21.15.5:0/727958608 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f353406d2d0 msgr2=0x7f35341304e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:35:09.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.067+0000 7f3532ffd640 1 --2- 172.21.15.5:0/727958608 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f353406d2d0 0x7f35341304e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:09.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.067+0000 7f3532ffd640 1 -- 172.21.15.5:0/727958608 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35340af9e0 con 0x7f353413f260 2024-03-18T15:35:09.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.067+0000 7f3533fff640 1 --2- 172.21.15.5:0/727958608 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f353414dc70 0x7f35340ae200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-18T15:35:09.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.067+0000 7f3532ffd640 1 --2- 172.21.15.5:0/727958608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f353413f260 0x7f35340adcc0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f351c0b0040 tx=0x7f351c019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:35:09.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.067+0000 7f352bfff640 1 -- 172.21.15.5:0/727958608 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351c0b2050 con 0x7f353413f260 2024-03-18T15:35:09.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.067+0000 7f352bfff640 1 -- 172.21.15.5:0/727958608 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f351c0b3040 con 0x7f353413f260 2024-03-18T15:35:09.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.067+0000 7f3539c5a640 1 -- 172.21.15.5:0/727958608 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f353413b660 con 0x7f353413f260 2024-03-18T15:35:09.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.067+0000 7f352bfff640 1 -- 172.21.15.5:0/727958608 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351c003dd0 con 0x7f353413f260 2024-03-18T15:35:09.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.067+0000 7f3539c5a640 1 -- 172.21.15.5:0/727958608 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35341e08a0 con 0x7f353413f260 2024-03-18T15:35:09.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.071+0000 7f3539c5a640 1 -- 172.21.15.5:0/727958608 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f353404f550 con 0x7f353413f260 2024-03-18T15:35:09.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.071+0000 7f352bfff640 1 -- 172.21.15.5:0/727958608 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 87083+0+0 (secure 0 0 0) 0x7f351c0c0020 con 0x7f353413f260 2024-03-18T15:35:09.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.071+0000 7f352bfff640 1 --2- 172.21.15.5:0/727958608 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3518067660 0x7f3518069b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-18T15:35:09.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.071+0000 7f352bfff640 1 -- 172.21.15.5:0/727958608 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 4943+0+0 (secure 0 0 0) 0x7f351c11fcd0 con 0x7f353413f260 2024-03-18T15:35:09.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.071+0000 7f35337fe640 1 --2- 172.21.15.5:0/727958608 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3518067660 0x7f3518069b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-18T15:35:09.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.071+0000 7f35337fe640 1 --2- 172.21.15.5:0/727958608 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3518067660 0x7f3518069b20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f35240015b0 tx=0x7f3524009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-18T15:35:09.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.075+0000 7f352bfff640 1 -- 172.21.15.5:0/727958608 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+177849 (secure 0 0 0) 0x7f351c016560 con 0x7f353413f260 2024-03-18T15:35:09.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.263+0000 7f3539c5a640 1 -- 172.21.15.5:0/727958608 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f353413c050 con 0x7f353413f260 2024-03-18T15:35:09.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.263+0000 7f352bfff640 1 -- 172.21.15.5:0/727958608 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+2216 (secure 0 0 0) 0x7f351c0f0bb0 con 0x7f353413f260 2024-03-18T15:35:09.269 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-03-18T15:35:09.269 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4619,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/2921781388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":2921781388},{"type":"v1","addr":"172.21.15.80:6833","nonce":2921781388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5268,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4160107376","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4160107376},{"type":"v1","addr":"172.21.15.5:6835","nonce":4160107376}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[]} 2024-03-18T15:35:09.269 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 18 2024-03-18T15:35:09.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.267+0000 7f3539c5a640 1 -- 172.21.15.5:0/727958608 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3518067660 msgr2=0x7f3518069b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:35:09.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.267+0000 7f3539c5a640 1 --2- 172.21.15.5:0/727958608 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3518067660 0x7f3518069b20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f35240015b0 tx=0x7f3524009040 comp rx=0 tx=0).stop 2024-03-18T15:35:09.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.267+0000 7f3539c5a640 1 -- 172.21.15.5:0/727958608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f353413f260 msgr2=0x7f35340adcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-18T15:35:09.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.267+0000 7f3539c5a640 1 --2- 172.21.15.5:0/727958608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f353413f260 0x7f35340adcc0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f351c0b0040 tx=0x7f351c019680 comp rx=0 tx=0).stop 2024-03-18T15:35:09.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.267+0000 7f3539c5a640 1 -- 172.21.15.5:0/727958608 shutdown_connections 2024-03-18T15:35:09.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.267+0000 7f3539c5a640 1 --2- 172.21.15.5:0/727958608 >> [v2:172.21.15.5:6832/3362034860,v1:172.21.15.5:6833/3362034860] conn(0x7f3518067660 0x7f3518069b20 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:09.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.267+0000 7f3539c5a640 1 --2- 172.21.15.5:0/727958608 >> [v2:172.21.15.80:3301/0,v1:172.21.15.80:6790/0] conn(0x7f353414dc70 0x7f35340ae200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:09.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.267+0000 7f3539c5a640 1 --2- 172.21.15.5:0/727958608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f353413f260 0x7f35340adcc0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:09.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.267+0000 7f3539c5a640 1 --2- 172.21.15.5:0/727958608 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f353406d2d0 0x7f35341304e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-18T15:35:09.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.267+0000 7f3539c5a640 1 -- 172.21.15.5:0/727958608 >> 172.21.15.5:0/727958608 conn(0x7f35340b5670 msgr2=0x7f353413c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-18T15:35:09.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.267+0000 7f3539c5a640 1 -- 172.21.15.5:0/727958608 shutdown_connections 2024-03-18T15:35:09.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-03-18T15:35:09.267+0000 7f3539c5a640 1 -- 172.21.15.5:0/727958608 wait complete. 2024-03-18T15:35:09.287 DEBUG:tasks.cephfs.filesystem:already dead... 2024-03-18T15:35:09.287 INFO:teuthology.misc:Shutting down mds daemons... 2024-03-18T15:35:09.287 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-18T15:35:09.287 INFO:teuthology.orchestra.run:waiting for 300 2024-03-18T15:35:09.296 INFO:tasks.ceph.mds.a:Stopped 2024-03-18T15:35:09.296 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-18T15:35:09.296 INFO:teuthology.orchestra.run:waiting for 300 2024-03-18T15:35:09.310 INFO:tasks.ceph.mds.b:Stopped 2024-03-18T15:35:09.310 INFO:teuthology.misc:Shutting down osd daemons... 2024-03-18T15:35:09.311 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-03-18T15:35:09.311 INFO:teuthology.orchestra.run:waiting for 300 2024-03-18T15:35:13.250 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-03-18T15:35:14.685 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-18T15:35:14.688 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-18T15:35:15.412 INFO:tasks.ceph.osd.0:Stopped 2024-03-18T15:35:15.413 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-03-18T15:35:15.413 INFO:teuthology.orchestra.run:waiting for 300 2024-03-18T15:35:15.470 INFO:tasks.ceph.osd.1:Stopped 2024-03-18T15:35:15.470 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-03-18T15:35:15.470 INFO:teuthology.orchestra.run:waiting for 300 2024-03-18T15:35:15.565 INFO:tasks.ceph.osd.2:Stopped 2024-03-18T15:35:15.566 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-03-18T15:35:15.566 INFO:teuthology.orchestra.run:waiting for 300 2024-03-18T15:35:19.156 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-03-18T15:35:19.157 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-03-18T15:35:21.667 INFO:tasks.ceph.osd.3:Stopped 2024-03-18T15:35:21.668 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-03-18T15:35:21.668 INFO:teuthology.orchestra.run:waiting for 300 2024-03-18T15:35:21.732 INFO:tasks.ceph.osd.4:Stopped 2024-03-18T15:35:21.732 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-03-18T15:35:21.732 INFO:teuthology.orchestra.run:waiting for 300 2024-03-18T15:35:21.799 INFO:tasks.ceph.osd.5:Stopped 2024-03-18T15:35:21.799 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-03-18T15:35:21.799 INFO:teuthology.orchestra.run:waiting for 300 2024-03-18T15:35:24.761 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-03-18T15:35:24.762 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-03-18T15:35:27.900 INFO:tasks.ceph.osd.6:Stopped 2024-03-18T15:35:27.901 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-03-18T15:35:27.901 INFO:teuthology.orchestra.run:waiting for 300 2024-03-18T15:35:27.956 INFO:tasks.ceph.osd.7:Stopped 2024-03-18T15:35:27.956 INFO:teuthology.misc:Shutting down mgr daemons... 2024-03-18T15:35:27.956 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-03-18T15:35:27.957 INFO:teuthology.orchestra.run:waiting for 300 2024-03-18T15:35:27.986 INFO:tasks.ceph.mgr.y:Stopped 2024-03-18T15:35:27.986 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-03-18T15:35:27.986 INFO:teuthology.orchestra.run:waiting for 300 2024-03-18T15:35:28.014 INFO:tasks.ceph.mgr.x:Stopped 2024-03-18T15:35:28.014 INFO:teuthology.misc:Shutting down mon daemons... 2024-03-18T15:35:28.014 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-03-18T15:35:28.014 INFO:teuthology.orchestra.run:waiting for 300 2024-03-18T15:35:28.044 INFO:tasks.ceph.mon.a:Stopped 2024-03-18T15:35:28.044 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-03-18T15:35:28.044 INFO:teuthology.orchestra.run:waiting for 300 2024-03-18T15:35:28.069 INFO:tasks.ceph.mon.b:Stopped 2024-03-18T15:35:28.070 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-18T15:35:28.070 INFO:teuthology.orchestra.run:waiting for 300 2024-03-18T15:35:28.132 INFO:tasks.ceph.mon.c:Stopped 2024-03-18T15:35:28.132 INFO:tasks.ceph:Checking cluster log for badness... 2024-03-18T15:35:28.133 DEBUG:teuthology.orchestra.run.smithi005:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'mons down, quorum' | egrep -v 'out of quorum' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-03-18T15:35:28.151 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi005.front.sepia.ceph.com 2024-03-18T15:35:28.151 DEBUG:teuthology.orchestra.run.smithi005:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-03-18T15:35:28.403 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi005.front.sepia.ceph.com 2024-03-18T15:35:28.403 DEBUG:teuthology.orchestra.run.smithi005:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-03-18T15:35:28.610 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi005.front.sepia.ceph.com 2024-03-18T15:35:28.610 DEBUG:teuthology.orchestra.run.smithi005:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-03-18T15:35:28.767 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi005.front.sepia.ceph.com 2024-03-18T15:35:28.767 DEBUG:teuthology.orchestra.run.smithi005:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-03-18T15:35:28.919 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi080.front.sepia.ceph.com 2024-03-18T15:35:28.919 DEBUG:teuthology.orchestra.run.smithi080:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-03-18T15:35:29.169 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi080.front.sepia.ceph.com 2024-03-18T15:35:29.169 DEBUG:teuthology.orchestra.run.smithi080:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-03-18T15:35:29.307 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi080.front.sepia.ceph.com 2024-03-18T15:35:29.307 DEBUG:teuthology.orchestra.run.smithi080:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-03-18T15:35:29.423 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi080.front.sepia.ceph.com 2024-03-18T15:35:29.423 DEBUG:teuthology.orchestra.run.smithi080:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-03-18T15:35:29.549 INFO:tasks.ceph:Archiving mon data... 2024-03-18T15:35:29.550 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608878/data/mon.a.tgz 2024-03-18T15:35:29.550 DEBUG:teuthology.orchestra.run.smithi005:> mktemp 2024-03-18T15:35:29.556 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/tmp.NO9s2x9WFV 2024-03-18T15:35:29.557 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.NO9s2x9WFV 2024-03-18T15:35:29.666 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0666 /tmp/tmp.NO9s2x9WFV 2024-03-18T15:35:29.731 DEBUG:teuthology.orchestra.remote:smithi005:/tmp/tmp.NO9s2x9WFV is 278KB 2024-03-18T15:35:29.763 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-03-18T15:35:29.791 DEBUG:teuthology.orchestra.run.smithi005:> rm -fr /tmp/tmp.NO9s2x9WFV 2024-03-18T15:35:29.797 DEBUG:teuthology.misc:Transferring archived files from smithi080:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608878/data/mon.b.tgz 2024-03-18T15:35:29.797 DEBUG:teuthology.orchestra.run.smithi080:> mktemp 2024-03-18T15:35:29.802 INFO:teuthology.orchestra.run.smithi080.stdout:/tmp/tmp.G88J3hklOX 2024-03-18T15:35:29.803 DEBUG:teuthology.orchestra.run.smithi080:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.G88J3hklOX 2024-03-18T15:35:29.929 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod 0666 /tmp/tmp.G88J3hklOX 2024-03-18T15:35:29.993 DEBUG:teuthology.orchestra.remote:smithi080:/tmp/tmp.G88J3hklOX is 373KB 2024-03-18T15:35:30.052 DEBUG:teuthology.orchestra.run.smithi080:> rm -fr /tmp/tmp.G88J3hklOX 2024-03-18T15:35:30.058 DEBUG:teuthology.misc:Transferring archived files from smithi080:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608878/data/mon.c.tgz 2024-03-18T15:35:30.058 DEBUG:teuthology.orchestra.run.smithi080:> mktemp 2024-03-18T15:35:30.110 INFO:teuthology.orchestra.run.smithi080.stdout:/tmp/tmp.70CQSKvkY9 2024-03-18T15:35:30.110 DEBUG:teuthology.orchestra.run.smithi080:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.70CQSKvkY9 2024-03-18T15:35:30.243 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod 0666 /tmp/tmp.70CQSKvkY9 2024-03-18T15:35:30.311 DEBUG:teuthology.orchestra.remote:smithi080:/tmp/tmp.70CQSKvkY9 is 373KB 2024-03-18T15:35:30.328 DEBUG:teuthology.orchestra.run.smithi080:> rm -fr /tmp/tmp.70CQSKvkY9 2024-03-18T15:35:30.333 INFO:tasks.ceph:Cleaning ceph cluster... 2024-03-18T15:35:30.333 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-18T15:35:30.335 DEBUG:teuthology.orchestra.run.smithi080:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-18T15:35:30.394 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-18T15:35:30.434 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-18T15:35:30.434 INFO:tasks.ceph:Archiving crash dumps... 2024-03-18T15:35:30.436 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608878/remote/smithi005/crash 2024-03-18T15:35:30.437 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-18T15:35:30.451 DEBUG:teuthology.misc:Transferring archived files from smithi080:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608878/remote/smithi080/crash 2024-03-18T15:35:30.451 DEBUG:teuthology.orchestra.run.smithi080:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-18T15:35:30.490 INFO:tasks.ceph:Shutting down logrotate 2024-03-18T15:35:30.490 DEBUG:teuthology.orchestra.run.smithi005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-18T15:35:30.495 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-18T15:35:30.579 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-18T15:35:30.593 DEBUG:teuthology.orchestra.run.smithi080:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-18T15:35:30.604 INFO:tasks.ceph:Compressing logs... 2024-03-18T15:35:30.604 DEBUG:teuthology.orchestra.run.smithi005:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-18T15:35:30.639 DEBUG:teuthology.orchestra.run.smithi080:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-18T15:35:30.668 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27616.log 2024-03-18T15:35:30.669 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27496.log 2024-03-18T15:35:30.669 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.27616.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-03-18T15:35:30.670 INFO:teuthology.orchestra.run.smithi080.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.27616.log.gz 2024-03-18T15:35:30.670 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.27496.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-03-18T15:35:30.670 INFO:teuthology.orchestra.run.smithi080.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.27496.log.gz 2024-03-18T15:35:30.670 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27520.log 2024-03-18T15:35:30.670 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-osd.7.log: /var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27592.log 2024-03-18T15:35:30.672 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.27520.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-03-18T15:35:30.673 INFO:teuthology.orchestra.run.smithi080.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.27520.log.gz 2024-03-18T15:35:30.673 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.27592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27640.log 2024-03-18T15:35:30.673 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-osd.5.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.27592.log.gz 2024-03-18T15:35:30.673 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-03-18T15:35:30.673 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.27640.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-18T15:35:30.673 INFO:teuthology.orchestra.run.smithi080.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.27640.log.gz 2024-03-18T15:35:30.673 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27472.log 2024-03-18T15:35:30.673 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-03-18T15:35:30.680 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.27472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27568.log 2024-03-18T15:35:30.680 INFO:teuthology.orchestra.run.smithi080.stderr: 87.7%/var/log/ceph/ceph-mon.c.log: -- replaced with /var/log/ceph/ceph-client.admin.27472.log.gz 2024-03-18T15:35:30.680 INFO:teuthology.orchestra.run.smithi080.stderr: 91.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-18T15:35:30.680 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-18T15:35:30.680 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.27568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27544.log 2024-03-18T15:35:30.680 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-03-18T15:35:30.681 INFO:teuthology.orchestra.run.smithi080.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.27568.log.gz 2024-03-18T15:35:30.681 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.27544.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.27544.log.gz 2024-03-18T15:35:30.681 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-03-18T15:35:30.681 INFO:teuthology.orchestra.run.smithi080.stderr: 92.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-03-18T15:35:30.681 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-03-18T15:35:30.681 INFO:teuthology.orchestra.run.smithi080.stderr: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-18T15:35:30.687 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35923.log 2024-03-18T15:35:30.687 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33983.log 2024-03-18T15:35:30.687 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.35923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32293.log 2024-03-18T15:35:30.687 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8%/var/log/ceph/ceph-client.admin.33983.log: -- replaced with /var/log/ceph/ceph-client.admin.35923.log.gz 2024-03-18T15:35:30.687 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33900.log 2024-03-18T15:35:30.688 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32293.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-03-18T15:35:30.688 INFO:teuthology.orchestra.run.smithi005.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33983.log.gz 2024-03-18T15:35:30.688 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.32293.log.gz 2024-03-18T15:35:30.688 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.33900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34139.log 2024-03-18T15:35:30.688 INFO:teuthology.orchestra.run.smithi005.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33900.log.gz 2024-03-18T15:35:30.688 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31400.log 2024-03-18T15:35:30.688 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33320.log 2024-03-18T15:35:30.688 INFO:teuthology.orchestra.run.smithi005.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34139.log.gz 2024-03-18T15:35:30.688 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36081.log 2024-03-18T15:35:30.689 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.31400.log: /var/log/ceph/ceph-client.admin.33320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33632.log 2024-03-18T15:35:30.689 INFO:teuthology.orchestra.run.smithi005.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.31400.log.gz 2024-03-18T15:35:30.689 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.36081.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33320.log.gz 2024-03-18T15:35:30.689 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32298.log 2024-03-18T15:35:30.689 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36081.log.gz 2024-03-18T15:35:30.689 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.33632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32616.log 2024-03-18T15:35:30.689 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32298.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33632.log.gz 2024-03-18T15:35:30.689 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32305.log 2024-03-18T15:35:30.689 INFO:teuthology.orchestra.run.smithi005.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.32298.log.gz 2024-03-18T15:35:30.690 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34987.log 2024-03-18T15:35:30.690 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32616.log: /var/log/ceph/ceph-client.admin.32305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33346.log 2024-03-18T15:35:30.690 INFO:teuthology.orchestra.run.smithi005.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32616.log.gz 2024-03-18T15:35:30.690 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34987.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.32305.log.gz 2024-03-18T15:35:30.690 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-03-18T15:35:30.690 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.33346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34243.log 2024-03-18T15:35:30.690 INFO:teuthology.orchestra.run.smithi005.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.34987.log.gz 2024-03-18T15:35:30.690 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-osd.0.log: 88.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.33346.log.gz 2024-03-18T15:35:30.691 INFO:teuthology.orchestra.run.smithi005.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.33268.log 2024-03-18T15:35:30.691 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34372.log 2024-03-18T15:35:30.691 INFO:teuthology.orchestra.run.smithi005.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34243.log.gz 2024-03-18T15:35:30.691 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.33268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31834.log 2024-03-18T15:35:30.691 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34450.log 2024-03-18T15:35:30.691 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34372.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33268.log.gz 2024-03-18T15:35:30.691 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.31834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34476.log 2024-03-18T15:35:30.691 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34372.log.gz 2024-03-18T15:35:30.691 INFO:teuthology.orchestra.run.smithi005.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.31834.log.gz 2024-03-18T15:35:30.692 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5/var/log/ceph/ceph-client.admin.34450.log: --verbose -- /var/log/ceph/ceph-client.admin.32590.log 2024-03-18T15:35:30.692 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35702.log 2024-03-18T15:35:30.692 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34450.log.gz 2024-03-18T15:35:30.692 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32590.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34476.log.gz 2024-03-18T15:35:30.692 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33216.log 2024-03-18T15:35:30.692 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32590.log.gz 2024-03-18T15:35:30.692 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.35702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33691.log 2024-03-18T15:35:30.692 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.33216.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35702.log.gz 2024-03-18T15:35:30.692 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34061.log 2024-03-18T15:35:30.693 INFO:teuthology.orchestra.run.smithi005.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33216.log.gz 2024-03-18T15:35:30.693 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.33691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27488.log 2024-03-18T15:35:30.693 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34061.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33691.log.gz 2024-03-18T15:35:30.693 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35605.log 2024-03-18T15:35:30.693 INFO:teuthology.orchestra.run.smithi005.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34061.log.gz 2024-03-18T15:35:30.693 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31647.log 2024-03-18T15:35:30.693 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.27488.log: gzip -5 --verbose --/var/log/ceph/ceph-client.admin.35605.log: /var/log/ceph/ceph-client.admin.35449.log 2024-03-18T15:35:30.693 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.27488.log.gz 2024-03-18T15:35:30.694 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-osd.4.log: 90.3% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-03-18T15:35:30.702 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.31647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33926.log 2024-03-18T15:35:30.702 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35605.log.gz 2024-03-18T15:35:30.702 INFO:teuthology.orchestra.run.smithi005.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.31647.log.gz 2024-03-18T15:35:30.702 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32044.log 2024-03-18T15:35:30.702 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.35449.log: /var/log/ceph/ceph-client.admin.33926.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35449.log.gz 2024-03-18T15:35:30.702 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35766.log 2024-03-18T15:35:30.702 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33926.log.gz 2024-03-18T15:35:30.703 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35819.log 2024-03-18T15:35:30.703 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32720.log 2024-03-18T15:35:30.703 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.35766.log: /var/log/ceph/ceph-client.admin.35819.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.32044.log.gz 2024-03-18T15:35:30.703 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-03-18T15:35:30.703 INFO:teuthology.orchestra.run.smithi005.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35766.log.gz 2024-03-18T15:35:30.703 INFO:teuthology.orchestra.run.smithi005.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35819.log.gz 2024-03-18T15:35:30.703 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35501.log 2024-03-18T15:35:30.703 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32720.log.gz 2024-03-18T15:35:30.703 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34113.log 2024-03-18T15:35:30.704 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-osd.2.log: /var/log/ceph/ceph-client.admin.35501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32642.log 2024-03-18T15:35:30.704 INFO:teuthology.orchestra.run.smithi005.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35501.log.gz 2024-03-18T15:35:30.704 INFO:teuthology.orchestra.run.smithi005.stderr:gzip/var/log/ceph/ceph-client.admin.34113.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.31862.log 2024-03-18T15:35:30.704 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32642.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-03-18T15:35:30.704 INFO:teuthology.orchestra.run.smithi005.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34113.log.gz 2024-03-18T15:35:30.704 INFO:teuthology.orchestra.run.smithi005.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32642.log.gz 2024-03-18T15:35:30.704 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.31862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34961.log 2024-03-18T15:35:30.704 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.31862.log.gz 2024-03-18T15:35:30.704 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31497.log 2024-03-18T15:35:30.705 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35397.log 2024-03-18T15:35:30.705 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.31497.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34961.log.gz 2024-03-18T15:35:30.705 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34297.log 2024-03-18T15:35:30.705 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.31497.log.gz 2024-03-18T15:35:30.705 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.35397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32097.log 2024-03-18T15:35:30.705 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35037.log 2024-03-18T15:35:30.705 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35397.log.gz 2024-03-18T15:35:30.705 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34502.log 2024-03-18T15:35:30.706 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34297.log.gz 2024-03-18T15:35:30.706 INFO:teuthology.orchestra.run.smithi005.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.32097.log.gz 2024-03-18T15:35:30.706 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.35096.log 2024-03-18T15:35:30.706 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.35037.log: /var/log/ceph/ceph-client.admin.34502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34191.log 2024-03-18T15:35:30.706 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35037.log.gz 2024-03-18T15:35:30.706 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.35096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32018.log 2024-03-18T15:35:30.706 INFO:teuthology.orchestra.run.smithi005.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34502.log.gz 2024-03-18T15:35:30.706 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31595.log 2024-03-18T15:35:30.706 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32018.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34191.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32283.log 2024-03-18T15:35:30.707 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-03-18T15:35:30.707 INFO:teuthology.orchestra.run.smithi005.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32018.log.gz 2024-03-18T15:35:30.707 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.31595.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.31595.log.gz 2024-03-18T15:35:30.707 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35728.log 2024-03-18T15:35:30.707 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35262.log 2024-03-18T15:35:30.707 INFO:teuthology.orchestra.run.smithi005.stderr: 87.5%/var/log/ceph/ceph-client.admin.35728.log: -- replaced with /var/log/ceph/ceph-client.admin.32283.log.gz 2024-03-18T15:35:30.707 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35210.log 2024-03-18T15:35:30.707 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35728.log.gz 2024-03-18T15:35:30.708 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.35262.log: 93.2% -- replaced with /var/log/ceph/ceph-client.0.35096.log.gz 2024-03-18T15:35:30.708 INFO:teuthology.orchestra.run.smithi005.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35262.log.gz 2024-03-18T15:35:30.708 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32078.log 2024-03-18T15:35:30.708 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.35210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34165.log 2024-03-18T15:35:30.708 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1%/var/log/ceph/ceph-client.admin.32078.log: -- replaced with /var/log/ceph/ceph-client.admin.35210.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35154.log 2024-03-18T15:35:30.708 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-03-18T15:35:30.708 INFO:teuthology.orchestra.run.smithi005.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.32078.log.gz 2024-03-18T15:35:30.708 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35184.log 2024-03-18T15:35:30.719 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34165.log.gz 2024-03-18T15:35:30.720 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.35154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31808.log 2024-03-18T15:35:30.720 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.35184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31699.log 2024-03-18T15:35:30.720 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.31808.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35184.log.gz 2024-03-18T15:35:30.720 INFO:teuthology.orchestra.run.smithi005.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35154.log.gz 2024-03-18T15:35:30.720 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34217.log 2024-03-18T15:35:30.720 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35793.log 2024-03-18T15:35:30.720 INFO:teuthology.orchestra.run.smithi005.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.31808.log.gz 2024-03-18T15:35:30.720 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.31699.log: /var/log/ceph/ceph-client.admin.34217.log: gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.31699.log.gz -5 2024-03-18T15:35:30.721 INFO:teuthology.orchestra.run.smithi005.stderr: --verbose -- /var/log/ceph/ceph-client.admin.31756.log 2024-03-18T15:35:30.721 INFO:teuthology.orchestra.run.smithi005.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34217.log.gz 2024-03-18T15:35:30.721 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.35793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34326.log 2024-03-18T15:35:30.721 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4%/var/log/ceph/ceph-client.admin.31756.log: -- replaced with /var/log/ceph/ceph-client.admin.35793.log.gz 2024-03-18T15:35:30.721 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31467.log 2024-03-18T15:35:30.721 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34326.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.31756.log.gz 2024-03-18T15:35:30.721 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31888.log 2024-03-18T15:35:30.721 INFO:teuthology.orchestra.run.smithi005.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34326.log.gz 2024-03-18T15:35:30.721 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.31467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36237.log 2024-03-18T15:35:30.722 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.31888.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.31467.log.gz 2024-03-18T15:35:30.722 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34632.log 2024-03-18T15:35:30.722 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.36237.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.31888.log.gz 2024-03-18T15:35:30.722 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33848.log 2024-03-18T15:35:30.722 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36237.log.gz 2024-03-18T15:35:30.722 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31673.log 2024-03-18T15:35:30.722 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34632.log.gz/var/log/ceph/ceph-client.admin.33848.log: 2024-03-18T15:35:30.722 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35650.log 2024-03-18T15:35:30.723 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.31673.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33848.log.gz 2024-03-18T15:35:30.723 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35475.log 2024-03-18T15:35:30.723 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.31673.log.gz 2024-03-18T15:35:30.723 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.35650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32668.log 2024-03-18T15:35:30.723 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.35475.log: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33769.log 2024-03-18T15:35:30.723 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35650.log.gz 2024-03-18T15:35:30.723 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32668.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35475.log.gz 2024-03-18T15:35:30.723 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32824.log 2024-03-18T15:35:30.723 INFO:teuthology.orchestra.run.smithi005.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.32668.log.gz 2024-03-18T15:35:30.724 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-03-18T15:35:30.724 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.33769.log: /var/log/ceph/ceph-client.admin.32824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35977.log 2024-03-18T15:35:30.724 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1%/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32098.log 2024-03-18T15:35:30.724 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32824.log.gz 2024-03-18T15:35:30.724 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33769.log.gz 2024-03-18T15:35:30.724 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.35977.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35977.log.gz 2024-03-18T15:35:30.724 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34580.log 2024-03-18T15:35:30.724 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36107.log 2024-03-18T15:35:30.724 INFO:teuthology.orchestra.run.smithi005.stderr: 88.9%/var/log/ceph/ceph-client.admin.34580.log: -- replaced with /var/log/ceph/ceph-client.admin.32098.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33372.log 2024-03-18T15:35:30.725 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-03-18T15:35:30.725 INFO:teuthology.orchestra.run.smithi005.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34580.log.gz 2024-03-18T15:35:30.725 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.36107.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36107.log.gz 2024-03-18T15:35:30.725 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32096.log 2024-03-18T15:35:30.725 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.33372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31782.log 2024-03-18T15:35:30.725 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33372.log.gz/var/log/ceph/ceph-client.admin.32096.log: 2024-03-18T15:35:30.725 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32091.log 2024-03-18T15:35:30.725 INFO:teuthology.orchestra.run.smithi005.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.32096.log.gz 2024-03-18T15:35:30.725 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.31782.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.31782.log.gz 2024-03-18T15:35:30.726 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36318.log 2024-03-18T15:35:30.726 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32694.log 2024-03-18T15:35:30.726 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3%/var/log/ceph/ceph-client.admin.36318.log: -- replaced with /var/log/ceph/ceph-client.admin.32091.log.gz 2024-03-18T15:35:30.726 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-03-18T15:35:30.726 INFO:teuthology.orchestra.run.smithi005.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36318.log.gz 2024-03-18T15:35:30.726 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32694.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.32694.log.gz 2024-03-18T15:35:30.726 INFO:teuthology.orchestra.run.smithi005.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-03-18T15:35:30.726 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33658.log 2024-03-18T15:35:30.727 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33138.log 2024-03-18T15:35:30.727 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-osd.1.log: /var/log/ceph/ceph-client.admin.33658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36029.log 2024-03-18T15:35:30.727 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4%/var/log/ceph/ceph-client.admin.33138.log: -- replaced with /var/log/ceph/ceph-client.admin.33658.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36342.log 2024-03-18T15:35:30.727 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-03-18T15:35:30.727 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33138.log.gz 2024-03-18T15:35:30.727 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34805.log 2024-03-18T15:35:30.727 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.36029.log: /var/log/ceph/ceph-client.admin.36342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36289.log 2024-03-18T15:35:30.728 INFO:teuthology.orchestra.run.smithi080.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-03-18T15:35:30.743 INFO:teuthology.orchestra.run.smithi005.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36029.log.gz 2024-03-18T15:35:30.743 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34805.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36342.log.gz 2024-03-18T15:35:30.743 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34087.log 2024-03-18T15:35:30.744 INFO:teuthology.orchestra.run.smithi005.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34805.log.gz 2024-03-18T15:35:30.744 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.36289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32876.log 2024-03-18T15:35:30.744 INFO:teuthology.orchestra.run.smithi005.stderr: 88.2%/var/log/ceph/ceph-client.admin.34087.log: -- replaced with /var/log/ceph/ceph-client.admin.36289.log.gz 2024-03-18T15:35:30.744 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36211.log 2024-03-18T15:35:30.744 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34087.log.gz 2024-03-18T15:35:30.744 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33502.log 2024-03-18T15:35:30.744 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36133.log 2024-03-18T15:35:30.744 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.36211.log: 90.9%/var/log/ceph/ceph-client.admin.33502.log: -- replaced with /var/log/ceph/ceph-client.admin.32876.log.gz 2024-03-18T15:35:30.744 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36211.log.gz 2024-03-18T15:35:30.745 INFO:teuthology.orchestra.run.smithi005.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33502.log.gz 2024-03-18T15:35:30.745 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34883.log 2024-03-18T15:35:30.745 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.36133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36159.log 2024-03-18T15:35:30.745 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34883.log: gzip 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36133.log.gz -5 2024-03-18T15:35:30.745 INFO:teuthology.orchestra.run.smithi005.stderr: --verbose -- /var/log/ceph/ceph-client.admin.34269.log 2024-03-18T15:35:30.745 INFO:teuthology.orchestra.run.smithi005.stderr: 87.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31727.log 2024-03-18T15:35:30.745 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34883.log.gz 2024-03-18T15:35:30.745 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.36159.log: /var/log/ceph/ceph-client.admin.34269.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36159.log.gz 2024-03-18T15:35:30.746 INFO:teuthology.orchestra.run.smithi005.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34269.log.gz 2024-03-18T15:35:30.746 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31543.log 2024-03-18T15:35:30.746 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35371.log 2024-03-18T15:35:30.746 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.31727.log: /var/log/ceph/ceph-client.admin.31543.log: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33957.log 2024-03-18T15:35:30.746 INFO:teuthology.orchestra.run.smithi005.stderr: -- replaced with /var/log/ceph/ceph-client.admin.31727.log.gz 2024-03-18T15:35:30.746 INFO:teuthology.orchestra.run.smithi005.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.31543.log.gz 2024-03-18T15:35:30.746 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.35371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36368.log 2024-03-18T15:35:30.746 INFO:teuthology.orchestra.run.smithi005.stderr: 89.4%/var/log/ceph/ceph-client.admin.33957.log: -- replaced with /var/log/ceph/ceph-client.admin.35371.log.gz 2024-03-18T15:35:30.747 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32095.log 2024-03-18T15:35:30.747 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.36368.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33957.log.gzgzip 2024-03-18T15:35:30.747 INFO:teuthology.orchestra.run.smithi005.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.33476.log 2024-03-18T15:35:30.747 INFO:teuthology.orchestra.run.smithi005.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36368.log.gz 2024-03-18T15:35:30.747 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35897.log 2024-03-18T15:35:30.747 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32095.log: /var/log/ceph/ceph-client.admin.33476.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-18T15:35:30.747 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.35897.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.32095.log.gz 2024-03-18T15:35:30.747 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33476.log.gz 2024-03-18T15:35:30.747 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33450.log 2024-03-18T15:35:30.748 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35897.log.gz 2024-03-18T15:35:30.748 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33874.log 2024-03-18T15:35:30.748 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.33450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32538.log 2024-03-18T15:35:30.748 INFO:teuthology.orchestra.run.smithi005.stderr: 88.9% 85.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-18T15:35:30.748 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.33874.log: -- replaced with /var/log/ceph/ceph-client.admin.33450.log.gz 2024-03-18T15:35:30.748 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35579.log 2024-03-18T15:35:30.748 INFO:teuthology.orchestra.run.smithi005.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33874.log.gz 2024-03-18T15:35:30.748 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32746.log 2024-03-18T15:35:30.748 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1%/var/log/ceph/ceph-client.admin.35579.log: -- replaced with /var/log/ceph/ceph-client.admin.32538.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32564.log 2024-03-18T15:35:30.749 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-03-18T15:35:30.749 INFO:teuthology.orchestra.run.smithi005.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35579.log.gz/var/log/ceph/ceph-client.admin.32746.log: 2024-03-18T15:35:30.749 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35423.log 2024-03-18T15:35:30.749 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32564.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.32746.log.gz 2024-03-18T15:35:30.749 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32564.log.gz 2024-03-18T15:35:30.749 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33112.log 2024-03-18T15:35:30.749 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34424.log 2024-03-18T15:35:30.749 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.35423.log: /var/log/ceph/ceph-client.admin.33112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32772.log 2024-03-18T15:35:30.750 INFO:teuthology.orchestra.run.smithi005.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35423.log.gz 2024-03-18T15:35:30.750 INFO:teuthology.orchestra.run.smithi005.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33112.log.gz 2024-03-18T15:35:30.750 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34424.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34424.log.gz 2024-03-18T15:35:30.750 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33294.log 2024-03-18T15:35:30.750 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35319.log 2024-03-18T15:35:30.750 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.33294.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.32772.log.gz 2024-03-18T15:35:30.750 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31940.log 2024-03-18T15:35:30.750 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33528.log 2024-03-18T15:35:30.751 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33294.log.gz 2024-03-18T15:35:30.751 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.35319.log: /var/log/ceph/ceph-client.admin.31940.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35319.log.gz 2024-03-18T15:35:30.751 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.31940.log.gz 2024-03-18T15:35:30.751 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34606.log 2024-03-18T15:35:30.751 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.33528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33606.log 2024-03-18T15:35:30.751 INFO:teuthology.orchestra.run.smithi005.stderr: 88.3%/var/log/ceph/ceph-client.admin.34606.log: -- replaced with /var/log/ceph/ceph-client.admin.33528.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35236.log 2024-03-18T15:35:30.751 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-03-18T15:35:30.751 INFO:teuthology.orchestra.run.smithi005.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34606.log.gz 2024-03-18T15:35:30.751 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.33606.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33606.log.gz 2024-03-18T15:35:30.752 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34688.log 2024-03-18T15:35:30.752 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.35236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33424.log 2024-03-18T15:35:30.752 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34688.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35236.log.gz 2024-03-18T15:35:30.752 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33822.log 2024-03-18T15:35:30.752 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34723.log 2024-03-18T15:35:30.752 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34688.log.gz/var/log/ceph/ceph-client.admin.33424.log: 2024-03-18T15:35:30.752 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.33822.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33424.log.gz 2024-03-18T15:35:30.752 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33822.log.gz 2024-03-18T15:35:30.752 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33164.log 2024-03-18T15:35:30.753 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33554.log 2024-03-18T15:35:30.753 INFO:teuthology.orchestra.run.smithi005.stderr: 89.0%/var/log/ceph/ceph-client.admin.33164.log: -- replaced with /var/log/ceph/ceph-client.admin.34723.log.gzgzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-18T15:35:30.753 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-03-18T15:35:30.753 INFO:teuthology.orchestra.run.smithi005.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33164.log.gz 2024-03-18T15:35:30.753 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.33554.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33554.log.gz 2024-03-18T15:35:30.753 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33398.log 2024-03-18T15:35:30.753 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33795.log 2024-03-18T15:35:30.753 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.33398.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-03-18T15:35:30.754 INFO:teuthology.orchestra.run.smithi080.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-03-18T15:35:30.764 INFO:teuthology.orchestra.run.smithi005.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33398.log.gz/var/log/ceph/ceph-client.admin.33795.log: 2024-03-18T15:35:30.765 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33795.log.gz 2024-03-18T15:35:30.765 INFO:teuthology.orchestra.run.smithi005.stderr: 89.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-18T15:35:30.765 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35288.log 2024-03-18T15:35:30.765 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31992.log 2024-03-18T15:35:30.765 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.35288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32306.log 2024-03-18T15:35:30.765 INFO:teuthology.orchestra.run.smithi005.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35288.log.gz 2024-03-18T15:35:30.765 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.31992.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.31992.log.gz 2024-03-18T15:35:30.765 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34909.log 2024-03-18T15:35:30.766 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32294.log 2024-03-18T15:35:30.766 INFO:teuthology.orchestra.run.smithi005.stderr: 87.9%/var/log/ceph/ceph-client.admin.34909.log: -- replaced with /var/log/ceph/ceph-client.admin.32306.log.gz 2024-03-18T15:35:30.766 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34909.log.gz 2024-03-18T15:35:30.766 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33717.log 2024-03-18T15:35:30.766 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35345.log 2024-03-18T15:35:30.766 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8%/var/log/ceph/ceph-client.admin.33717.log: -- replaced with /var/log/ceph/ceph-client.admin.32294.log.gz 2024-03-18T15:35:30.766 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33717.log.gz 2024-03-18T15:35:30.766 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32931.log 2024-03-18T15:35:30.767 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.35345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34660.log 2024-03-18T15:35:30.767 INFO:teuthology.orchestra.run.smithi005.stderr: 89.4%/var/log/ceph/ceph-client.admin.32931.log: -- replaced with /var/log/ceph/ceph-client.admin.35345.log.gz 2024-03-18T15:35:30.767 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32931.log.gz 2024-03-18T15:35:30.767 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32092.log 2024-03-18T15:35:30.767 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31621.log 2024-03-18T15:35:30.767 INFO:teuthology.orchestra.run.smithi005.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34660.log.gz 2024-03-18T15:35:30.767 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32092.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.32092.log.gz 2024-03-18T15:35:30.767 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.33062.log 2024-03-18T15:35:30.767 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.31621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36185.log 2024-03-18T15:35:30.768 INFO:teuthology.orchestra.run.smithi005.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.31621.log.gz 2024-03-18T15:35:30.768 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.33062.log: 89.4% -- replaced with /var/log/ceph/ceph-client.0.33062.log.gz 2024-03-18T15:35:30.768 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35676.log 2024-03-18T15:35:30.768 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.36185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27269.log 2024-03-18T15:35:30.768 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4%/var/log/ceph/ceph-client.admin.35676.log: -- replaced with /var/log/ceph/ceph-client.admin.36185.log.gz 2024-03-18T15:35:30.768 INFO:teuthology.orchestra.run.smithi005.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35676.log.gz 2024-03-18T15:35:30.768 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31569.log 2024-03-18T15:35:30.768 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.27269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36003.log 2024-03-18T15:35:30.768 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.31569.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.27269.log.gz 2024-03-18T15:35:30.769 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.31569.log.gz 2024-03-18T15:35:30.769 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32297.log 2024-03-18T15:35:30.769 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.36003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34528.log 2024-03-18T15:35:30.769 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32297.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36003.log.gz 2024-03-18T15:35:30.769 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.32297.log.gz 2024-03-18T15:35:30.769 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34398.log 2024-03-18T15:35:30.769 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33190.log 2024-03-18T15:35:30.769 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34398.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34528.log.gz 2024-03-18T15:35:30.770 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34398.log.gz 2024-03-18T15:35:30.770 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34009.log 2024-03-18T15:35:30.770 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.33190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31966.log 2024-03-18T15:35:30.770 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33190.log.gz 2024-03-18T15:35:30.770 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34009.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34009.log.gz 2024-03-18T15:35:30.770 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34831.log 2024-03-18T15:35:30.770 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.31966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32512.log 2024-03-18T15:35:30.770 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34831.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.31966.log.gz 2024-03-18T15:35:30.770 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34831.log.gz 2024-03-18T15:35:30.771 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32486.log 2024-03-18T15:35:30.771 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27378.log 2024-03-18T15:35:30.779 INFO:teuthology.orchestra.run.smithi005.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.32512.log.gz/var/log/ceph/ceph-client.admin.32486.log: 2024-03-18T15:35:30.780 INFO:teuthology.orchestra.run.smithi005.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.32486.log.gz 2024-03-18T15:35:30.780 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.22507.log 2024-03-18T15:35:30.780 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.27378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33580.log 2024-03-18T15:35:30.780 INFO:teuthology.orchestra.run.smithi005.stderr: 87.8%/var/log/ceph/ceph.tmp-client.admin.22507.log: -- replaced with /var/log/ceph/ceph-client.admin.27378.log.gz 2024-03-18T15:35:30.780 INFO:teuthology.orchestra.run.smithi005.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.22507.log.gz 2024-03-18T15:35:30.780 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34779.log 2024-03-18T15:35:30.780 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.33580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35871.log 2024-03-18T15:35:30.780 INFO:teuthology.orchestra.run.smithi005.stderr: 87.6%/var/log/ceph/ceph-client.admin.34779.log: -- replaced with /var/log/ceph/ceph-client.admin.33580.log.gz 2024-03-18T15:35:30.781 INFO:teuthology.orchestra.run.smithi005.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34779.log.gz 2024-03-18T15:35:30.781 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34753.log 2024-03-18T15:35:30.781 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.35871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31914.log 2024-03-18T15:35:30.781 INFO:teuthology.orchestra.run.smithi005.stderr: 88.5%/var/log/ceph/ceph-client.admin.34753.log: -- replaced with /var/log/ceph/ceph-client.admin.35871.log.gz 2024-03-18T15:35:30.781 INFO:teuthology.orchestra.run.smithi005.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34753.log.gz 2024-03-18T15:35:30.781 INFO:teuthology.orchestra.run.smithi005.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-03-18T15:35:30.781 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35845.log 2024-03-18T15:35:30.781 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32296.log 2024-03-18T15:35:30.781 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.31914.log: /var/log/ceph/ceph-client.admin.35845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34857.log 2024-03-18T15:35:30.782 INFO:teuthology.orchestra.run.smithi005.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.31914.log.gz 2024-03-18T15:35:30.782 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4%/var/log/ceph/ceph-client.admin.32296.log: -- replaced with /var/log/ceph/ceph-client.admin.35845.log.gz 2024-03-18T15:35:30.782 INFO:teuthology.orchestra.run.smithi005.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.32296.log.gz 2024-03-18T15:35:30.782 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35951.log 2024-03-18T15:35:30.782 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34035.log 2024-03-18T15:35:30.782 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.35951.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34857.log.gz 2024-03-18T15:35:30.782 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36055.log 2024-03-18T15:35:30.782 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33242.log 2024-03-18T15:35:30.783 INFO:teuthology.orchestra.run.smithi005.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35951.log.gz 2024-03-18T15:35:30.783 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34035.log: /var/log/ceph/ceph-client.admin.36055.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34035.log.gz 2024-03-18T15:35:30.783 INFO:teuthology.orchestra.run.smithi005.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36055.log.gz 2024-03-18T15:35:30.783 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35553.log 2024-03-18T15:35:30.783 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.33242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34935.log 2024-03-18T15:35:30.783 INFO:teuthology.orchestra.run.smithi005.stderr: 88.7%/var/log/ceph/ceph-client.admin.35553.log: -- replaced with /var/log/ceph/ceph-client.admin.33242.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32088.log 2024-03-18T15:35:30.783 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-03-18T15:35:30.783 INFO:teuthology.orchestra.run.smithi005.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35553.log.gz 2024-03-18T15:35:30.783 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.34935.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34935.log.gz 2024-03-18T15:35:30.784 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32850.log 2024-03-18T15:35:30.784 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33743.log 2024-03-18T15:35:30.784 INFO:teuthology.orchestra.run.smithi005.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.32088.log.gz/var/log/ceph/ceph-client.admin.32850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35527.log 2024-03-18T15:35:30.784 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-03-18T15:35:30.784 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32850.log.gz 2024-03-18T15:35:30.784 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.33743.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33743.log.gz 2024-03-18T15:35:30.784 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32798.log 2024-03-18T15:35:30.784 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.35527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36263.log 2024-03-18T15:35:30.784 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.32798.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35527.log.gz 2024-03-18T15:35:30.785 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34554.log 2024-03-18T15:35:30.785 INFO:teuthology.orchestra.run.smithi005.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32798.log.gz 2024-03-18T15:35:30.785 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.admin.36263.log: /var/log/ceph/ceph-client.admin.34554.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36263.log.gz 2024-03-18T15:35:30.785 INFO:teuthology.orchestra.run.smithi005.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34554.log.gz 2024-03-18T15:35:30.785 INFO:teuthology.orchestra.run.smithi080.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-03-18T15:35:30.792 INFO:teuthology.orchestra.run.smithi005.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-03-18T15:35:30.798 INFO:teuthology.orchestra.run.smithi080.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-03-18T15:35:30.799 INFO:teuthology.orchestra.run.smithi005.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-03-18T15:35:30.800 INFO:teuthology.orchestra.run.smithi080.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-03-18T15:35:30.826 INFO:teuthology.orchestra.run.smithi080.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-03-18T15:35:30.827 INFO:teuthology.orchestra.run.smithi080.stderr: 2024-03-18T15:35:30.827 INFO:teuthology.orchestra.run.smithi080.stderr:real 0m0.178s 2024-03-18T15:35:30.827 INFO:teuthology.orchestra.run.smithi080.stderr:user 0m0.654s 2024-03-18T15:35:30.827 INFO:teuthology.orchestra.run.smithi080.stderr:sys 0m0.053s 2024-03-18T15:35:30.831 INFO:teuthology.orchestra.run.smithi005.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-03-18T15:35:30.848 INFO:teuthology.orchestra.run.smithi005.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-03-18T15:35:30.888 INFO:teuthology.orchestra.run.smithi005.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-03-18T15:35:30.889 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-03-18T15:35:30.889 INFO:teuthology.orchestra.run.smithi005.stderr:real 0m0.248s 2024-03-18T15:35:30.889 INFO:teuthology.orchestra.run.smithi005.stderr:user 0m0.900s 2024-03-18T15:35:30.890 INFO:teuthology.orchestra.run.smithi005.stderr:sys 0m0.103s 2024-03-18T15:35:30.890 INFO:tasks.ceph:Archiving logs... 2024-03-18T15:35:30.890 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608878/remote/smithi005/log 2024-03-18T15:35:30.891 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-18T15:35:31.177 DEBUG:teuthology.misc:Transferring archived files from smithi080:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608878/remote/smithi080/log 2024-03-18T15:35:31.178 DEBUG:teuthology.orchestra.run.smithi080:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-18T15:35:31.263 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-18T15:35:31.273 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-18T15:35:31.273 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-18T15:35:31.276 DEBUG:teuthology.orchestra.run.smithi080:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-18T15:35:31.352 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-18T15:35:31.361 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-18T15:35:31.361 DEBUG:teuthology.orchestra.run.smithi005:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-18T15:35:31.364 DEBUG:teuthology.orchestra.run.smithi080:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-18T15:35:31.375 INFO:teuthology.orchestra.run.smithi005.stdout: remote refid st t when poll reach delay offset jitter 2024-03-18T15:35:31.376 INFO:teuthology.orchestra.run.smithi005.stdout:============================================================================== 2024-03-18T15:35:31.376 INFO:teuthology.orchestra.run.smithi005.stdout:*hv01.front.sepi 44.4.53.2 2 u 37 64 77 0.103 -2.527 1.231 2024-03-18T15:35:31.376 INFO:teuthology.orchestra.run.smithi005.stdout:+hv02.front.sepi 63.231.80.2 3 u 41 64 77 0.217 +0.990 1.267 2024-03-18T15:35:31.376 INFO:teuthology.orchestra.run.smithi005.stdout:+hv03.front.sepi 74.6.168.72 3 u 45 64 77 0.128 -0.791 0.963 2024-03-18T15:35:31.376 INFO:teuthology.orchestra.run.smithi005.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-18T15:35:31.378 INFO:teuthology.orchestra.run.smithi080.stdout: remote refid st t when poll reach delay offset jitter 2024-03-18T15:35:31.378 INFO:teuthology.orchestra.run.smithi080.stdout:============================================================================== 2024-03-18T15:35:31.378 INFO:teuthology.orchestra.run.smithi080.stdout:*hv01.front.sepi 44.4.53.2 2 u 44 64 77 0.130 -0.651 3.877 2024-03-18T15:35:31.378 INFO:teuthology.orchestra.run.smithi080.stdout:+hv02.front.sepi 63.231.80.2 3 u 43 64 77 0.075 +2.962 3.935 2024-03-18T15:35:31.378 INFO:teuthology.orchestra.run.smithi080.stdout:+hv03.front.sepi 74.6.168.72 3 u 43 64 77 0.124 +1.134 3.935 2024-03-18T15:35:31.378 INFO:teuthology.orchestra.run.smithi080.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-18T15:35:31.379 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-18T15:35:31.388 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-18T15:35:31.388 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-18T15:35:31.397 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-18T15:35:31.406 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-18T15:35:31.427 INFO:teuthology.task.internal:Duration was 808.346110 seconds 2024-03-18T15:35:31.427 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-18T15:35:31.477 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-18T15:35:31.477 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-18T15:35:31.480 DEBUG:teuthology.orchestra.run.smithi080:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-18T15:35:31.517 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-18T15:35:31.518 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi005.front.sepia.ceph.com 2024-03-18T15:35:31.518 DEBUG:teuthology.orchestra.run.smithi005:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-18T15:35:31.575 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi080.front.sepia.ceph.com 2024-03-18T15:35:31.575 DEBUG:teuthology.orchestra.run.smithi080:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-18T15:35:31.585 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-18T15:35:31.585 DEBUG:teuthology.orchestra.run.smithi005:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-18T15:35:31.619 DEBUG:teuthology.orchestra.run.smithi080:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-18T15:35:31.659 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-18T15:35:31.659 DEBUG:teuthology.orchestra.run.smithi005:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-18T15:35:31.698 DEBUG:teuthology.orchestra.run.smithi080:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-18T15:35:31.757 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-18T15:35:31.772 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-18T15:35:31.772 DEBUG:teuthology.orchestra.run.smithi005:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-18T15:35:31.785 DEBUG:teuthology.orchestra.run.smithi080:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-18T15:35:31.797 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-18T15:35:31.808 DEBUG:teuthology.orchestra.run.smithi005:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-18T15:35:31.831 DEBUG:teuthology.orchestra.run.smithi080:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-18T15:35:31.840 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern = core 2024-03-18T15:35:31.848 INFO:teuthology.orchestra.run.smithi080.stdout:kernel.core_pattern = core 2024-03-18T15:35:31.991 DEBUG:teuthology.orchestra.run.smithi005:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-18T15:35:31.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T15:35:31.996 DEBUG:teuthology.orchestra.run.smithi080:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-18T15:35:32.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-18T15:35:32.038 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-18T15:35:32.050 INFO:teuthology.task.internal:Transferring archived files... 2024-03-18T15:35:32.051 DEBUG:teuthology.misc:Transferring archived files from smithi005:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608878/remote/smithi005 2024-03-18T15:35:32.051 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-18T15:35:32.073 DEBUG:teuthology.misc:Transferring archived files from smithi080:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608878/remote/smithi080 2024-03-18T15:35:32.073 DEBUG:teuthology.orchestra.run.smithi080:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-18T15:35:32.101 INFO:teuthology.task.internal:Removing archive directory... 2024-03-18T15:35:32.102 DEBUG:teuthology.orchestra.run.smithi005:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-18T15:35:32.115 DEBUG:teuthology.orchestra.run.smithi080:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-18T15:35:32.146 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-18T15:35:32.157 INFO:teuthology.task.internal:Not uploading archives. 2024-03-18T15:35:32.157 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-18T15:35:32.166 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-18T15:35:32.167 DEBUG:teuthology.orchestra.run.smithi005:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-18T15:35:32.169 DEBUG:teuthology.orchestra.run.smithi080:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-18T15:35:32.173 INFO:teuthology.orchestra.run.smithi005.stdout: 658468 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 18 15:35 /home/ubuntu/cephtest 2024-03-18T15:35:32.195 INFO:teuthology.orchestra.run.smithi080.stdout: 658641 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 18 15:35 /home/ubuntu/cephtest 2024-03-18T15:35:32.196 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-18T15:35:32.207 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-18T15:35:32.256 INFO:teuthology.nuke:Checking targets against current locks 2024-03-18T15:35:32.281 DEBUG:teuthology.nuke:shortname: smithi005 2024-03-18T15:35:32.281 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-18T15:35:32.308 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi005.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608878', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-18 15:12:57.349464', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBeylqcswqUr05yIIhATKJxEzhYbd6C3wDv2PdDWT4RmCi7iiC/LBqC9AtG2GTM3aparYmIWuzcG5frRFfBO4DI='} 2024-03-18T15:35:32.309 DEBUG:teuthology.nuke:shortname: smithi080 2024-03-18T15:35:32.309 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-18T15:35:32.333 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi080.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-03-18_14:19:46-fs-wip-batrick-testing-20240314.183243-distro-default-smithi/7608878', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-18 15:12:57.347407', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHNmff7IMarXwxiL+qz9KB3L6ZwbgMuqx413Lxw7UxAhVWj+9yFYOxhyKc9j8ckv7x6x9TGTIXs7Jwujqc59FpQ='} 2024-03-18T15:35:32.370 INFO:teuthology.orchestra.console.smithi005:Power off 2024-03-18T15:35:32.370 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-18T15:35:32.395 DEBUG:teuthology.orchestra.console.smithi005:power off output: Chassis Power Control: Down/Off 2024-03-18T15:35:32.395 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T15:35:32.412 INFO:teuthology.orchestra.console.smithi080:Power off 2024-03-18T15:35:32.412 DEBUG:teuthology.orchestra.console.smithi080:pexpect command: ipmitool -H smithi080.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-18T15:35:32.422 DEBUG:teuthology.orchestra.console.smithi005:check power output: Chassis Power is on 2024-03-18T15:35:32.435 DEBUG:teuthology.orchestra.console.smithi080:power off output: Chassis Power Control: Down/Off 2024-03-18T15:35:32.435 DEBUG:teuthology.orchestra.console.smithi080:pexpect command: ipmitool -H smithi080.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T15:35:32.458 DEBUG:teuthology.orchestra.console.smithi080:check power output: Chassis Power is on 2024-03-18T15:35:36.423 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T15:35:36.460 DEBUG:teuthology.orchestra.console.smithi080:pexpect command: ipmitool -H smithi080.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T15:35:36.534 DEBUG:teuthology.orchestra.console.smithi005:check power output: Chassis Power is on 2024-03-18T15:35:36.570 DEBUG:teuthology.orchestra.console.smithi080:check power output: Chassis Power is on 2024-03-18T15:35:40.536 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T15:35:40.571 DEBUG:teuthology.orchestra.console.smithi080:pexpect command: ipmitool -H smithi080.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T15:35:40.646 DEBUG:teuthology.orchestra.console.smithi005:check power output: Chassis Power is on 2024-03-18T15:35:40.681 DEBUG:teuthology.orchestra.console.smithi080:check power output: Chassis Power is on 2024-03-18T15:35:44.651 DEBUG:teuthology.orchestra.console.smithi005:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T15:35:44.681 DEBUG:teuthology.orchestra.console.smithi080:pexpect command: ipmitool -H smithi080.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-18T15:35:44.762 DEBUG:teuthology.orchestra.console.smithi005:check power output: Chassis Power is off 2024-03-18T15:35:44.790 DEBUG:teuthology.orchestra.console.smithi080:check power output: Chassis Power is off 2024-03-18T15:35:44.863 INFO:teuthology.orchestra.console.smithi005:Power off completed 2024-03-18T15:35:44.892 INFO:teuthology.orchestra.console.smithi080:Power off completed 2024-03-18T15:35:44.996 INFO:teuthology.run:Summary data: description: fs/shell/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/ubuntu_latest mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/cephfs-shell} duration: 808.3461096286774 failure_reason: 'Test failure: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD)' flavor: default owner: scheduled_pdonnell@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=08087e8ed7a5413ab8e57fdadb076eb4 status: fail success: false 2024-03-18T15:35:44.996 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-18T15:35:45.078 INFO:teuthology.run:FAIL